Users Aid in AI Advancements
This is the story of the rebirth of contextual targeting and incorporating natural language processing, all thanks to the patience of internal users.
Background
Preparing for the end of the tracking pixel with contextual targeting
The tracking pixels' days were numbered due to privacy concerns.
Product and engineering identified contextual targeting, a traditional method that places digital ads based on a webpage's content, to future-proof the display-side platform against the uncertainty surrounding tracking pixels.
Users must be able to define a context matching eligible URLs for ad placement.
Short-term problems
Defining the context is a task that should be carried out with client input.
The pre-sales tool was not equipped to handle the context object.
The responsibility of creating the context was pushed to a different role, which requires detailed information about the specific tactic, leading to a lack of confidence in creating the context.
Milestones in the journey
Engineer built to start
In the very beginning, engineers close to the underlying technology built and edited all the contexts.
Context Builder 1.0 ships
Users started building contexts for campaigns in the UI with engineers providing support.
Adding NLP and user feedback
Engineers took the discoveries of 1.0 and user feedback supercharged it with word embeddings and natural language processing.
Context Builder 2.0 ships
A streamlined front-end the setup and improving the accuracy of the resulting placements the targeting method produces.
UX Taking a different role
More focus on the back-end service
The components and interactions of Context Builder 1.0 and 2.0 were relatively simple. Most of the iteration was in an effort to alleviate user frustrations by pushing the capabilities of the service to provide insights about the context that was created.
Consoling frustrated users
The research in this project was the heaviest lift for the UX group. Numerous user interviews were conducted over the two years of this project. These turned into necessary efforts to keep users engaged and, at the very least, let them know we were still listening.
UI WORK
Context builder 1.0 card
The first version was made with established inputs and patterns from the design system.
This mimicked everything engineering was using in their Jupyter Notebook.
We tested with users, but at this stage, there wasn't much we could change.
Engineering and data science continued to analyze the performance of the tactic in live campaigns.
Users' key feedback was that they were not confident in entering criteria to define the context.
Users had two lingering issues
What combination of criteria makes a good context?
How do we know if it's a good context?
Engineering gets inspired
The engineering groups got the data they needed, and it looked promising.
They started pushing hard on word embeddings and natural language processing, which meant fewer inputs could produce exponentially better results.
Pushing on what the service could provide back to the user
I designed concepts that gave users feedback on whether a context was good or not. Every way I did this caused more configuration steps, plus engineers told us the backend service could not break down its results in this way.
Top URLs and Apps for the win
We aligned on the idea of providing a list of top URLs and apps, giving a sense of what pages the client's ads would be placed. Combined with the relevant URL and daily impressions estimations, it gave users enough to assess the context.
Context builder 2.0 card
Inputs:
Streamlined inputs made for quicker and less restrictive setup.
The context definition is the single input for the natural language processing engine.
Users are no longer restricted by a rigid category taxonomy
Estimations:
Quantitative estimations are now provided.
Exposing the top URLs and Apps serves as qualitative feedback for the user for a better understanding of what the context will produce.
Results
Context builder is ready to be placed in other tools
We solved the problem of the context builder being too complicated. The back-end service that engineering created for Context Builder 2.0 was filed and received a patent.
The context builder still needs to be included in pre-sales tools. That, however, was going to be part of a much larger project.
Looking back
Recognizing our super users bridging the gap
The unfortunate circumstances users faced inspired me to push all aspects of the context builder, ensuring it could provide as much value to users as possible. I admired the dedication our users had throughout the process. They truly bridged the gap to get the context builder to a modern state using the latest technology.
Team & Culture
Leading a shift in our team's dynamics, which positively impacted our culture and performance
Identifying the Need to Overhaul
Investigating the shortcomings of systems that were straining the business and uncovering the extent of necessary changes.