Is my AV crashing? An online photo-based experiment assessing whether shared intended pathway can help AV drivers anticipate silent failures

Figures & data

Table 1. Photo experiment 2 × 2 design conditions.

Independent variables	HMI 1 (pathway-only)	HMI 2 (pathway + object recognition)
Crash (pathway showing unsafe manoeuvre)
Safe (pathway showing safe manoeuvre)

Figure 1. Photos from the experiment showing the two HMI 2 (Full HUD) conditions.

Two pictures are displayed side-by-side. On the left is an unsafe manoeuvre: a silent failure or potential crash ahead, as the pathway indicates that the ego-vehicle will not stop and give way to the car on the roundabout. On the right is a safe manoeuvre: no potential crash ahead, as the pathway indicates that the ego-vehicle will stop and give way to the car on the roundabout.

Figure 2. In the top left photo showing the pathway as green, participants were explained: ‘If the car intends to ACCELERATE (maintain speed or go faster), a GREEN pathway will be shown’. In the bottom left photo showing the pathway as yellow, participants were explained: ‘If the car intends to CRUISE (neither accelerate nor brake), a YELLOW pathway will be shown’. In the right-hand photo where the pathway is shown as red. Participants were explained: ‘If the car intends to brake (slow down and/or stop), a red pathway will be shown’.

Three pictures are shown from the perspective of a driver of vehicle. The vehicle is on an exit ramp from a motorway and approaching a motorway. The top left picture shows the car while it is far away from the roundabout, and is still accelerating up the exit ramp. The SIP is green and continues straight to follow the lane. The bottom-left picture shows the vehicle as it is closer to the approach of the roundabout. The SIP is yellow as the car is coasting and slowing down. The right-hand picture is showing the vehicle as it is about to stop behind a stopped car waiting at the entrance to the roundabout. The SIP is red showing that the vehicle will brake and stop behind the stopped car.

Figure 3. Photo showing the object recognition bounding boxes. Participants were explained: ‘If the car has seen another road user, say a pedestrian or another vehicle; or a traffic sign or traffic lights, it will show a BLUE BOX around it, as shown in the picture above’.

The picture shows the same roundabout as. There are two cars and trucks entering the roundabout ahead of the vehicle. They have the blue bounding boxes placed around them indicating that the AV sensors have detected them. The Traffic light and Give Way sign also have bounding boxes surrounding them.

Figure 4. The photos used to explain an unsafe manoeuvre (left) and a safe manoeuvre (right).

Read the detailed description of this figure

Two pictures are displayed side-by-side. They are both identical, except for the SIP that is presented. The photo shows the perspective of a driver of a vehicle waiting, giving way to a truck which is on the main road. On the left version of the photo, the SIP is showing that the car intends to accelerate into the truck, as the SIP is green and is directed into the truck. This is an example of where human intervention would be necessary to avoid an unsafe manoeuvre. On the right version, the SIP is showing that the car will brake and come to a stop, giving way to the truck, as the SIP is red and ends at the give way road marking. In this case, human intervention is not necessary to avoid an unsafe manoeuvre.

Figure 5. The photos used for the trials used in the experiment.

On the left, the first photo trial where an intervention is necessary because the pathway shows that AV intends to accelerate and cut across the right-hand side lane, potentially crashing with the vehicle on the right-hand side. On the right-hand side, is the second photo trial. Human intervention is not necessary here because the pathway shows the AV intending to slop at the red traffic lights in a safe manner.

Figure 6. The same traffic scenario shown in the four different conditions. Photos in the left column are ‘safe’ manoeuvres while those in the right column are ‘unsafe’ manoeuvres. The top row includes object recognition bounding boxes whereas the bottom row does not.

There are four images, presented 2 × 2. Other than the ORBBs and SIP, they are all identical, as it shows, from the perspective of a driver, a car approaching a roundabout where a red car is already on the roundabout. In the left column, the SIP is red and ends at the give way line. In the right column, the SIP is green and continues around the roundabout. The top two pictures contain ORBBs, while the bottom two do not.