Why recommendation apps ask you to rate things first

Open any app that promises to tell you what to watch, read or listen to next and it will ask you something before it does anything useful. Rate these ten movies. Pick three artists. Choose the genres you like.

That step is easy to read as onboarding friction, a form standing between you and the product. It is closer to the opposite. It is the product asking the only question it will ever get a free answer to, and most apps waste it.

Why it cannot just start

An app with no history for you has exactly one thing to rank by, which is what other people did.

That is not useless. Popularity is a reasonable guess in the absence of anything better, and for a brand new user it is genuinely the best available answer. The trouble is that it is also the answer you already have. Every catalogue on your television already leads with what is popular, and if a recommendation app can only tell you that Movie Of The Month is doing well, you have installed a second copy of the problem.

So the app has to get from nothing to something specific about you, and it has to do it before you lose interest. That is the cold start problem, and it is a question design problem long before it is a data problem. You cannot have somebody’s history before they arrive. You can decide what to ask them.

Making the question cheaper changes everything

The clearest evidence that the format matters more than the theory comes from Netflix.

For years they collected five-star ratings. In 2016 they tested a straight thumbs up or thumbs down against it, and the number of ratings people gave went up by 200 percent. They switched the whole product over in March 2017.

Nothing about anybody’s taste changed that year. The only thing that changed was how much thought one answer cost. A five-star scale asks you to decide whether a movie was a three or a four, which is a question most people cannot answer about a movie they liked, so they skip it. Thumbs asks whether you liked it. Tripling your data by making the question easier is a better return than almost any modelling improvement.

The question everybody asks, and why it lies

The most common opening question is some version of “pick a few things you like”.

It performs well in testing, because it is fast and it feels good, and it produces a worse profile than it looks like it should. People do not name what they actually watch. They name what they would like to be the kind of person who watches. The list comes back heavy on the important movie they saw once and light on the comfort watch they have been through four times, and the app dutifully learns the version of you that you presented rather than the one on the sofa.

This is why an app that watches what you do usually beats an app that asks what you like, and why the short questionnaire has to be treated as a rough sketch rather than a portrait. It is a direction, not a description.

A random sample is the intuitive answer and a bad one

Say you have decided to ask, and you need a set of things to ask about. The obvious approach is to draw them at random from the catalogue.

It fails twice. The first is that you will not have heard of most of them. A large catalogue is mostly long tail, so a fair random sample is dominated by titles almost nobody has seen, and ten rounds of “no idea” teaches the app nothing while teaching you that it has no idea either.

The second is quieter. A random sample inherits whatever the catalogue is heavy on. If a third of it is recent American genre movies, roughly a third of your questions will be too, and those titles are all near neighbours. Rating six of them tells an app not much more than rating one. You have spent six questions asking the same question.

What a good question actually looks like

Two things make a title worth asking about, and they pull against each other.

It should be far away from the other things you have been asked, because a title unlike everything else on the list is one whose answer cannot be guessed from the answers already given. And it should be recognisable, because you cannot rate what you have never seen. Those requirements are in tension by nature, since the titles everybody knows are the popular ones, and popular titles resemble each other. Whatever any app does here is a compromise, not a solution.

Then there is the part that surprises people. A movie almost everyone loves is a bad question.

If ninety-five people in a hundred rate something highly, your answer was predictable before you gave it, and a predictable answer carries almost no information about you specifically. The valuable questions are the ones people split on. Something half an audience adores and half cannot sit through is a genuine fork in the road, and which side you land on says something the crowd average never could. That is also the honest reason a good opening set often feels like an odd list rather than a run of obvious classics.

What none of this fixes

An opening questionnaire cannot tell the difference between “I do not like this” and “this was never put in front of me”.

Your answers describe what you have been exposed to as much as what you enjoy, and anyone whose taste sits outside the well-known middle starts further from it than everyone else and has to work their way back. Filtering for recognisability makes that worse, because a familiarity bar is a popularity bar with better manners, and popularity is not spread evenly across countries or decades.

There is no clever fix. There is only being honest that the first answers are a sketch, and building the thing so the sketch gets corrected quickly by what you actually do next.

Where we land on it

For what it is worth, Filmatic asks for about twenty movies, chosen to be ones most people will recognise while still being spread out rather than twenty versions of the same movie. Twenty is not a magic number. It is roughly where one more answer stops being worth the tap.

The part that matters more is what happens after. If you have read how the recommendations work, the profile those twenty answers produce is a starting position and nothing more. Every rating after it moves the thing, which is the whole reason an opening questionnaire is allowed to be imperfect.