This Riddler puzzle is about randomly generating convex quadrilaterals.

Consider four square-shaped ranches, arranged in a two-by-two pattern, as if part of a larger checkerboard. One family lives on each ranch, and each family builds a small house independently at a random place within the property. Later, as the families in adjacent quadrants become acquainted, they construct straight-line paths between the houses that go across the boundaries between the ranches, four in total. These paths form a quadrilateral circuit path connecting all four houses. This circuit path is also the boundary of the area where the families’ children are allowed to roam.

What is the probability that the children are able to travel in a straight line from any allowed place to any other allowed place without leaving the boundaries? (In other words, what is the probability that the quadrilateral is convex?)

Here is my solution:

[Show Solution]

The only way the quadrilateral can be nonconvex is if one of its vertices is “inward”. In other words, an internal angle of the quadrilateral is greater than 180 degrees. The figure below shows an example of a nonconvex case (on the left) and a convex case (on the right). For the nonconvex example, the vertex labeled K is inward.

It’s clear that at most one vertex can be inward. So, by symmetry, we can write the following statement regarding the probability of the quadrilateral being convex.

\begin{align}

\mathbb{P}(\text{convex}) &= 1-\mathbb{P}(\text{nonconvex}) \\

&= 1-\mathbb{P}(\text{one vertex is inward})\\

&= 1-4\cdot\mathbb{P}(\text{a particular vertex is inward})

\end{align}So let’s calculate the probability of a particular vertex being inward. This only depends on the positions of the two adjacent vertices. Consider the diagram below and opposite vertices $J(-a,b)$ and $L(p,-q)$ where $0 \le a,b,p,q \le 1$ are the coordinates. The vertex $K(u,v)$ with $0 \le u,v \le 1$ will be inward if it ends up in the shaded triangle. The probability of this occurring is equal to the area of the triangle.

Since we’d like to find the total probability of $K$ being inward, we must find the expected area of the shaded triangle with respect to the points $J$ and $L$ being uniformly randomly chosen in their respective regions. The area is $A=\tfrac{1}{2}xy$ where $x$ and $y$ are the intersections with the axes indicated. Calculating $x$ and $y$ amounts to finding the intersection of the line segment $JL$ with the axes. A bit of algebra reveals the formula:

\[

x = \frac{bp-aq}{b+q},

\qquad

y = \frac{bp-aq}{a+p}

\]To find the expected area, we integrate $\frac{1}{2}xy$ over $0\le a,b,p,q \le 1$. We must take care to account for the case where $x$ and $y$ are both negative, which we do by dividing the result by $2$. Finally, we substitute the result into the formula for the probability of convexity and we obtain:

\begin{align}

\mathbb{P}(\text{convex}) &= 1-4\cdot\mathbb{P}(\text{vertex }K\text{ is inward})\\

&= 1-4 \int_0^1\int_0^1\int_0^1\int_0^1 \frac{1}{2}\cdot\frac{1}{2}\,x\,y\,\,\mathrm{d}a\,\mathrm{d}b\,\mathrm{d}p\,\mathrm{d}q\\

&= 1-\int_0^1\int_0^1\int_0^1\int_0^1 \frac{(bp-aq)^2}{(a+p)(b+q)}\,\mathrm{d}a\,\mathrm{d}b\,\mathrm{d}p\,\mathrm{d}q\\

&=\frac{1}{6}(11-8\log2)\\

&\approx 0.909137

\end{align}So the probability that the quadrilateral is convex is about 90.9%.

My Monte Carlo code estimated the probability to be 0.9091(4), in agreement with your result.

https://gist.github.com/ShiangYong/c4db131c18b171e4c857dc9327c0f412