Channel your inner Cornholio

You’re playing a game of cornhole with your friends, and it’s your turn to toss the four bean bags. For every bean bag you toss onto your opponents’ board, you get $1$ point. For every bag that goes through the hole on their board, you get $3$ points. And for any bags that don’t land on the board or through the hole, you get $0$ points.

Your opponents had a terrible round, missing the board with all their throws. Meanwhile, your team currently has $18$ points — just $3$ points away from victory at $21.$ You’re also playing with a special house rule: To win, you must score exactly $21$ points, without going over.

Based on your history, you know there are three kinds of throws you can make: (1) An aggressive throw, which has a $40$ percent chance of going in the hole, a $30$ percent chance of landing on the board and a $30$ percent chance of missing the board and hole; (2) a conservative throw, which has a $10$ percent chance of going in the hole, an $80$ percent chance of landing on the board and a $10$ percent chance of missing the board and hole; or (3) a wasted throw, which has a $100$ percent chance of missing the board and hole.

For each bean bag, you can choose any of these three throws. Your goal is to maximize your chances of scoring exactly 3 points with your four tosses. What is the probability that your team will finish the round with exactly 21 points and declare victory?

Let's denote the three types of throw strategies as $A$ for aggressive, $C$ for conservative and $W$ for wasted. Let $S_k$ be the score after the $k$th throw of this turn (with $S_0=18$). Based on the strategy, the score will evolve probabilistically according to probability laws $\mathbb{P}_A,$ $\mathbb{P}_C$ or $\mathbb{P}_W,$ respectively, as described above. We only care about ending up with $S_4 = 21$ and should set $V(S) = \delta(S-21) = \begin{cases} 1, &\text{if $S=21$;}\\ 0, &\text{if $S \ne 21.$}\end{cases}$$We can also think of the score in terms of five states$\mathcal{S} = \{18, 19, 20, 21, \gt 21\}$ to make the problem more manageable.

If we start with the final utility $V_4 = V$ as above, then working backwards, we get $$V_{k-1}(s) = \max_u \sum_{\sigma \in \mathcal{S}} \mathbb{P}_u \{S_k=\sigma \mid S_{k-1} = s\} V_k(\sigma),$$ for each $s \in \mathcal{S}.$ Ultimately, we are looking for the value of $V_0(18),$ which we will use backwards induction to infer.

Intuitively, the optimal values should be $\hat{u}(21) = \hat{u}(\gt 21) = W,$ as in either case there is no additional benefit to scoring any more points (either you have already lost or would lose if any more points are scored). Then we will always have $V_k(21) = 1$ and $V_k(\gt 21)=0$ for all $k = 1, 2, 3, 4.$

Moving to $s=20,$ we get $$\begin{align*}V_{k-1}(20) &= \max \{ 0.3 V_k(20) + 0.3 V_k(21) + 0.4 V_k(\gt 21), 0.1 V_k(20) + 0.8 V_k(21) + 0.1 V_k(\gt 21), V_k(20) \}\\ &= \max\{ 0.3 V_k(20) + 0.3, 0.1 V_k(20) + 0.8, V_k(20) \},\end{align*}$$ with final condition $V_4(20) = 0.$ Therefore, we have $V_3(20) = 0.8,$ $V_2(20) = 0.88,$ and $V_1(20) = 0.888.$ Note that for $s=20,$ the optimal choice is always $\hat{u}(20)=C.$

Dropping down to $s=19,$ we get $$\begin{align*}V_{k-1}(19) &= \max \{0.3 V_k(19) + 0.3 V_k(20) + 0.4 V_k(\gt 21), 0.1 V_k(19) + 0.8 V_k(20) + 0.1 V_k(\gt 21), V_k(19)\} \\ &= \max \{0.3 V_k(19) + 0.3 V_k(20), 0.1 V_k(19) + 0.8 V_k(20), V_k(20) \},\end{align*}$$ with final condition $V_4(19) = 0.$ Therefore, we have $V_3(19) = 0,$ $V_2(19) = 0.64$, and $V_1(19) = 0.768.$ For $s=19,$ the optimal choice (except for $\hat{u}_4$ where all options are equally worthless) is $\hat{u}(19) = C.$

Moving to $s=18,$ we have $$\begin{align*}V_{k-1}(18) &= \max \{0.3 V_k(18) + 0.3 V_k(19) + 0.4 V_k(21), 0.1 V_k(18) + 0.8 V_k(19) + 0.1 V_k(21), V_k(18) \}\\ &= \max \{ 0.3 V_k(18) + 0.3 V_k(19) + 0.4, 0.1 V_k(18) + 0.8 V_k(19) + 0.1, V_k(18) \},\end{align*}$$ with final condition $V_4(18) = 0.$ Therefore we have $V_3(18) = 0.4$, $V_2(18) = 0.52$ and $V_1(18) = 0.748$ and $V_0(18) = 0.8548.$ Note again here that we also have $\hat{u}(18) = A$ regardless of the number of bean bags already tossed during your turn. So with a uniform strategy of agressive toss if you are currently at $S=18;$ a conservative toss if you are currently at $S \in \{19, 20\};$ and wasted toss if your score is $S \geq 21,$ we get the probability of winning during your next turn as $V_0(18) = 85.48\%$.