Use AI for Market and Competitor Intelligence at Scale

Keeping track on the market at large, and your competitors in particular, is a demanding task these days as competitors can be mentioned or leave digital footprints anywhere on the Internet.

AI-based tools are well positioned to harvest data from millions of webpages to surface valuable business insights. One such tool is Crayon, that automatically harvest market- and competitor intelligence using artificial intelligence.

I had the opportunity to discuss this with Ellie Mirman, who is the CMO at Crayon. Prior to Crayon, she was Vice President of Marketing at Toast, where she built and led the marketing function across demand generation, content marketing, product marketing, branding, and customer advocacy.

Now, over to the interview on gathering market- and competitor intelligence using artificial intelligence!

Crayon was founded in 2014 by former HubSpot and AdMob executives who believe that millions of businesses have yet to take advantage of the intelligence data available today to drive actionable insights and opportunities. 

Businesses today have great visibility internally – into their sales, marketing, and product metrics and strategies – but lack that visibility externally; Crayon is a market intelligence platform that helps businesses get that visibility into their markets, competitors, and customers.

Crayon is a market and competitive intelligence platform that ​enables businesses to track, analyze, and act on everything happening outside their four walls.

Crayon provides complete intelligence on competitors, ranging from product and pricing changes to customer and employee reviews to marketing campaigns and tests.

The platform tracks a competitor’s complete digital footprint, both on and off their website. This full range of intelligence allows a business to get an understanding of their competitors’ strategies and programs.

To collect all of this intelligence data, the Crayon platform taps into millions of intelligence sources, tracking each competitor’s moves programmatically and at scale.

Oftentimes, the most critical updates are those that aren’t announced in press releases but rather are hidden in subtle website changes or a new customer review. 

The data is automatically categorized and prioritized as well, to enable business users to review and analyze the most important data. With the incredible amount of intelligence data available, it’s critical to surface the highest priority updates to be able to analyze and act on them while they are still fresh.

Crayon uses a combination of machine learning and analyst curation to separate the signal from the noise in competitive intelligence.

Finally, the Crayon platform provides tools for collaboration, analysis, and action. Being able to interact with the data, discuss the latest updates, and view the data from different lenses allows cross-functional teams to spot competitive trends and devise an action plan based on intel.

Crayon uses artificial intelligence in two primary ways: (1) to identify meaningful competitive changes and (2) to separate the signal from noise in competitive intelligence data.

Crayon analyzes hundreds of millions of webpages to capture critical competitive moves wherever they happen. But not every change is meaningful – for example, out of tens of thousands of webpage changes captured by Crayon in the last year, less than two tenths of a percent of them were meaningful enough to surface to a company monitoring one of those companies.

By using AI, the platform is able to tell the difference between the correction of a typo on a homepage (not meaningful) versus the change of one number on a pricing page (meaningful).

There is a wealth of competitive intelligence available online, and AI can help highlight the most critical of updates among the meaningful changes.

By analyzing the content, type, source, and historical engagement of a specific competitive insight, Crayon can determine how likely it is that this insight is “signal” versus “noise.”

For example, a Quora thread that mentions three competitors is likely to be important to a company, especially if they participate in a lot of such discussion threads. The source (Quora), type (discussion thread), content (three competitors mentioned), and interaction (past participation in such discussions) provide overwhelming evidence that this insight is important to highlight.

The ideal place for AI in business is to complement and enhance humans. AI presents the opportunity for businesses to automate routine tasks so that humans can better use their time to provide higher value contributions.

AI also opens the door to tasks that are better suited for technology than humans, such as identifying subtle anomalies in large data sets.

Marketing will never be fully automated because people are not fully automated, and marketing involves communication with people.

More than ever, buyers desire human connections, so as AI becomes a bigger part of the marketing technology stack, there will need to be a significant place carved out for human interaction and connection.

No. AI has the potential to make marketers more efficient and more effective.

Firstly, AI can replace repetitive, programmable tasks so that marketers can focus their time and effort on more high value activities. For example, Crayon’s AI eliminates millions of datapoints that aren’t meaningful and don’t require a marketer to review, freeing up time for the marketer to analyze the datapoints that are relevant to their business.

AI can also help marketers be more effective by suggesting tactics that have a higher likelihood of success. For example, a content AI platform can suggest blog topics or headlines that have garnered higher-than-average results. Blog writers would then be able to level-up all of their efforts so that they see a higher level of success.