I had already established a roadmap for our internal tools to integrate machine learning, but I discovered a need to clearly broadcast the vision for how AI will work in our product. This could be considered a sort of product vision statement – north star for internal stakeholders and an easy to understand conceptual description for others (investors, partners, etc.).
The video portion is an approved clip from a partner. We run our object recognition system on each frame of it, producing bounding boxes and object tags. This is easily painted on each frame, giving a cool effect for each clothing items detected and tracked over time.
As the video progresses and new characters appear on screen, a new placeholder row appears for the character. It remains a silhouette until enough of the character’s face has been seen to identify who it is.
Clothes and Accessories
Similarly, when an object (clothing) is identified, a placeholder box appear next to that character’s headshot. A progress bar indicates how much of the product has been seen, allowing the system to know better what it is.
It eventually figures out: what kind of clothing it is, what brand, and the retail image (implies we know where you can buy it).
While it took some effort to make, the video was a success with everyone. This idea that the more it watches a video, the more frames it sees, it can progressively make better/more-certain conclusions about what’s appearing on screen – it works (maybe not the same as the actual system).