In this post I want to share some ideas to solve the challenges when you want to add high quality keywords for your products:
- It is very common that you have thousands of products. It is no option to do the work manually – so we have to automate the process somehow to be scalable
- Just concatenating columns in your product feed normally is a bad idea because most of the keywords will never get traffic: product names in your master data is not the way the users are searching. Keywords are getting too specific.
- If possible use keywords where no brand identifier of the product is used: The competition will be very low on those search terms.
- Use modified broad matches in the shortest possible way that matches to your product – this will avoid that you blow up your account with redundant keywords.
Let’s use the the shopping feed to build our own search engine
Sounds complicated – but after some hundred lines of code the results are already what I was looking for. The base concepts I’m using are:
- Frequencies on n-grams
- Stemming of words
- Mapping product ids to ngrams
Now I’m able to lookup whether a search query belongs to a unique product or not.
Important: Unique means in the scope of our product feed. It is still possible that the query matches to a lot of other products in the google search.
Feed our search engine with user searches from e.g. google shopping
The idea is very simple: Our users know how to search for products in relevant ways. In a lot of cases this is different compared to our product title in our feed.
In my example project I had a very big search query list. My workflow looks like this:
- For every query I first did entity extraction of the brand names of the product.
- The remaining string is the interesting part for generating our high quality keywords.
- Now we use this string to our search engine – if there is a unique matching product we return its productId.
- Use all combinations of our input string and send them also to the search engine to find the shortest possible n-gram keywords.
The result of this process looks already great – but there are still some keywords that are not specific in the scope of the google search. The lean approach is to just add the keywords and filter out all keywords with high impressions and bad CTRs after a while.
Adding some checks whether our keywords are specific and relevant
Again – the user queries are great for testing our new keywords. Remember: the keywords are specific when it comes to matching within our product feed.
My first check is this:
- Simulate matching of our new keyword set with all available search queries.
- Extract brand entities and count brands in search queries
- If more than one brand appears the keyword is not specific when it comes to user searches => not a valid keyword!
Another case I discovered is this: Even when the keyword is specific in the product feed and out of the users search query perspective (for our business) it is possible that we are still generic in the google world. My approach for this is:
- Send the keyword to googles autocomplete
- Run the entity recognition of brandnames on autocomplete results
- If we are able to extract the same brand that our matched product belongs to the keyword is relevant.
What if there is no user query list available?
This will be the case if you start with a new google ads account. A great free way to extract user searches is to send your product brand names to googles autocomplete and use the results in a recursive way to get the product longtail.