Hotwire-Cars Demand Validation Experiment
I ran a simple quasi-experiment with the goal of uncovering pain points in Hotwire’s booking process, specifically when consumers are planning trips abroad, i.e. Naples, Italy, and are looking for rental cars.
I followed the test requirement scoring system that Pauline Marol from Hotwire had outlined in her interview with Dan Siroker, before I began to structure my quasi-experiment. I wanted to know where in Hotwire’s “Test & Learn” backlog this test request would likely be ranked; thus, a score of 8/10 justified the next step of actually interviewing U.S. rental car consumers.
Problem
Currently, U.S. travelers looking for a rental car on Hotwire.com do not have the option to filter search results by a car’s transmission type, although the platform does label the cars’ transmission type. This can become a friction point in the user’s journey. Users could leave the platform for competitors* like Autoeurope, Holiday Autos, Europcar, and BingoCarRental, as well as suppliers like Hertz’s local divisions, which all have this feature currently available. This friction point in the customer’s journey can have implications that affect the product’s bottom line. The informed consumer is likely to switch to a competitor who can satisfy this specific need, but the average consumer is unlikely to think about a rental car’s transmission type and naturally assume the search result returns options that address their needs.
Objective
Conduct a qualitative quasi-experiment evaluating the demand for the option to filter search results by transmission type (i.e. manual or automatic cars).
After researching Hotwire and its sister brands, I inferred that Hotwire’s main metric is number of bookings and the secondary metric would likely be number of completed bookings in 60 seconds — similar to sister company CarRentals.com, which is not as effective as Hotwire.com in displaying automatic cars when returning search results for international destinations. However, because CarRentals.com has a larger inventory; and thus, the largest rental car selection in the world, adding this proposed feature could also serve beneficial to CarRentals’ bottom line.
My hypothesis: once participants searched for rental cars with the intent to drive it in a city abroad, i.e. Naples, Italy, they would prefer to refine the search results by a car’s transmission type.
Target Users
The most severe limitation of this quasi-experiment is an unrepresentative sample. I did not have access to the platform’s database, and as a result was unable to sample from the appropriate population. Because representative sampling is a challenge, I decided to cast participants from my personal network to observe Hotwire’s user experience (UX).
My participant selection criteria were: working professionals between the ages 25–44 who travel frequently (3 or more unique trips in a year) and rented a car on the last trip they took, either domestically or international and for either leisure or business purposes. Participants were not required to have used Hotwire.com prior to the quasi-experiment.
Methodology
It was a three-step process. After participants (n=5) were selected, I set down with them to establish context and explain the task.
The process was timed. (A) Searching for a car was limited to 30 seconds, and (B) walking through the test screen was also timed at 30 seconds. However, (C) the open-ended survey administered to participants at the end did not have a time limit, but on average it took about 2 minutes to complete.
Task
Participants were asked to plan a weeklong vacation in Naples, Italy where they would be driving a car for the duration of the trip. Participants were able to select any month but the vacation had to last 7 days. Once outstanding questions were answered, I presented them with a laptop that had the Hotwire.com homepage open** and instructed them to enter (1) Naples, Italy into the destination search bar and (2) select a 7-day date frame of their choice from Hotwire’s calendar.
Quasi-experiment parameters
1. Participants (n=5) conveniently sampled for the quasi-experiment.
2. N=5 completed the first phase and selected a car from search results. This task was timed and limited to 30 seconds.
3. After the first 30 seconds expired, participants (n=5) were shown a screen of a search result from Hotwire.com, which illustrated very similar information as the page they just engaged with. The only difference, besides the date frame of the results page, was that the page shown to participants contained an additional element simulating a feature to filter search results by transmission type.
This added element on the results page represented a filter option by automatic transmission type on a visible location above the fold. Then participants were asked to (1) walk through the screen and explain their thought process for selecting the car they had just previously picked, and (2) explain if the options available on this screen presented to them would change their search refinement process.
I did not draw their attention to the added elements nor did I consciously direct their behavior during the data collection period.
Questions I wanted to answer
I came up with the following three questions as a result of my initial Internet search for complaints voiced by U.S. consumers who were planning a trip abroad and were also looking for an automatic rental car.
1. When you book a rental car for a trip abroad, what is the first thing that you consider about the car that you are contemplating to book?
2. Think about all the cars that you have previously booked. Do you ever contemplate the transmission type of the cars you book for either domestic or international travel?
3. If the website where you are searching for a rental car had the option to filter for automatic and manual transmission cars, which type of car would you prefer to book?
User Feedback
The demand validation experiment returned expected results, but also pointed out additional friction points that were not specifically addressed in the post-experiment questionnaire.
Primary take away is the majority of participants (n=4) did not consider a rental car’s transmission type when they planned a trip to a European city where they hypothetically planned to drive a car. Only one participant paused during the first phase of the quasi-experiment and asked,
“How can I select an automatic car? Where is the filter for it? … I unfortunately don’t know how to drive a stick shift.”
The data, although not representative, suggests that U.S. consumers do not generally think about a rental car’s transmission type when they rent a car, even for trips abroad. While for three participants price was the most important concern, two participants pointed out that size and airport pickup and drop-off options were the first things they think about when searching for a rental car.
However, one participant expressed that the car should be similar to an American car.
“Whether I can drive the car, because … I look for cars that are familiar to what I know from America.”
To the question whether participants ever think about a rental car’s transmission type, all participants (n=5) said,
“No, … because I will automatically expect an automatic.”
All participants (n=5) declared that they do not know how to drive a car with a manual transmission and would prefer an automatic car. One participant identified the underpinning challenge and captured the participants‘ shared sentiment.
“I would prefer to book an automatic car, but not sure if that’s something that a lot of car renters really pay attention to. I automatically expect them to either be an automatic or have a dual function.”
This is a considerable friction point for the average U.S. consumer traveling abroad, where in the majority of countries automatic cars are rare and come at a premium. Implementing a solution would not only improve the UX of this particular use case, but also positively impact Hotwire’s bottom line.
Conclusion
Users generally expect a search for rental cars to return appropriate results, i.e. a reasonably priced automatic mid-sized sedan that they can pick up and drop-off at an airport. This exploratory quasi-experiment served its purpose in uncovering a friction point in the UX. The findings, in addition to the test requirements score of 8, justify further inquiry with a sample drawn from the appropriate population of Hotwire users who book cars for trips abroad.
I believe the next test should aim to ascertain whether these niche consumers would use the new filter — only a single participant in my quasi-experiment noticed the altered screen — or whether the solution should be implemented primarily in the backend, i.e. placing automatic cars on the top of the search result, instead of leading with manuals. Such a solution has the potential to either 1) increase revenue from existing users, i.e. default bookings of more expensive automatic cars in markets where these types of cars are scarce, or 2) reduce the bounce rate of users who are specifically looking for a rental car with an automatic transmission.
Expected outcomes of proposed A/B split test: (1) Increase bookings of automatic cars in international markets, and (2) reduce bounce rate of Hotwire.com users who are looking for automatic rental cars in cities abroad, where the default option is a manual transmission car.
My initial hypothesis has been somewhat validated; despite the unrepresentativeness of my sample, Hotwire users who book a rental car in a European city prefer to rent an automatic car. However, the next test should be more refined and investigate the following hypothesis:
If search results ranked automatic cars to the top of the results page, then more automatic cars would be rented by U.S. consumers travelling abroad and who are booking through Hotwire.com, because a 100% of participants (n=5) in a guerilla usability test stated that they would prefer to drive a familiar transmission type and 100% also indicated that they had never thought about a rental car’s transmission type before.
Hotwire’s site is beautifully designed. I love the company, and I love its products. I just wanted to be helpful and contribute to making the platform more powerful by exploring this edge case. I am not affiliated with Hotwire.
*I used the competitors’ products, and search results seemed to suggest that these sites have the same inventory as Hotwire (Autoeuro, Europcar, etc.).
**I used my iPhone to hotspot the Internet connection during the data collection period.
