Comment Analysis

Our Comment Analysis feature enables you to easily analyze your open-text feedback at scale. Capabilities include full text search, term expansion, sentiment analysis, trending topics, and visual reporting of sentiment and frequency. Comment Analysis only looks at comments from Milestone surveys and excludes comments from Exit, Always On and Point in Time Surveys.

Trending Topics

Trending Topics powered by Bright™ is a component of the Comment Analysis feature set that uses Large Language Models to identify topics that are surfacing through open-text feedback. In the platform, it will be referred to as Trending Topics along with a lightbulb icon to indicate this is a Bright™ feature.

To access Trending Topics, navigate to Feedback, then open the Comment Analysis tab:

Defining a Topic

A topic is a grouping of comments discussing the same or similar thing, even if the comments do not mention the topic specifically. A comment can have more than one topic assigned to it. It could also have no topic assigned to it if there isn’t one that is deemed relevant (for example a comment that says, “N/A”).

The list of Trending Topics for a company is the collection of most frequently discussed topics via comments. To qualify as a topic, it needs to be mentioned across at least 8 check-ins in the last 4 weeks across that company and that company needs to have at least 30 check-ins responded to in the last 4 weeks. 

Trending Topics powered by Bright™ assigns comments to a topic out of a list of hundreds (~350) of industry specific topics that have been defined through the analysis of historical comment data, manually reviewed, and curated. 

Understanding the Trending Topics Table

The Trending Topics table refreshes daily and includes topic frequency ranking, rank change, and sentiment calculated at a company level. 

• Topic frequency is measured by the unique number of check-ins that mention that topic over the last 4 weeks (28 days).
• Rank change is calculated by comparing the current 4 week period to a previous 4 week period offset by two weeks (ie: July 11 - Aug 8 compared to June 23 - July 25).
• The platform will show a maximum of 40 topics per company; there is no minimum number of topics.
• If a topic wasn’t previously on the list of topics and is entering the top 40 rank, it will be considered ranked as 40th for the purpose of calculating the rank change.
• The default sorting of the Trending Topics table is by frequency rank. You can sort by either the rank change and sentiment columns. Sorting on rank change will sort on the absolute value of the rank change.

You have the option to explore comments within your scope that are related to company-wide topics by clicking on the topic from the table. This will bring you to the full text search page, with more detailed reporting, where you can drill into relevant comments.

Managing Trending Topics

All users will be given access to Trending Topics powered by Bright™. Employers can choose to disable Trending Topics, while still maintaining access to the other Comment Analysis features. Disabling Trending Topics is done at a company level and cannot be disabled for specific users.

Trending Topics vs. Categories and Opportunities

• Alert categories are a subset of the topic whitelist, but not all topics are on the alert categories menu. The reasons for this today are:
  • Alerts highlight comments that are sensitive or urgent and not all topics fit that description.
  • Alert categories are customizable while Topics are not. 
• Topics are not the same as Opportunities in a few ways: 
  • Opportunities are based on the quantitative answers to questions, grouped by the question theme, while topics are based on the qualitative answer content, grouped by the answer content. 
  • Opportunities are grouped into general categories, but can be filtered for a user’s specific scope, while topics are more specific categories that will first be presented at a company level. 

Report Cards

Clicking on any row in the Trending Topics table or searching for a term will create Report card, which includes data on all comments surrounding that topic. The Report card also features filterable Overview, Sentiment, and Frequency charts. 

As an example, clicking into "Workload" from the Trending topics table or searching the term "Workload" will generate a Report Card containing all comments that pertain to workload in the last 180 days.

Full Text Search

Looking for comments related to a particular term? Input one or many keywords into the search bar to search comments over the last 180 days. Tip: When searching multiple keywords at once, separate the words by a space.

You'll be able to apply filters to narrow results either before you begin searching or after results have been returned. Note that all searches will be “OR” searches and the current version does not support exact searches or “AND” searches. An “AND” search would only populate results that include every word included in that search, while an “OR” search will populate results that include any of the words included in the search. For comments in languages other than English, the platform will search the translation of the comment.

Fuzzy Search

Fuzzy search functionality refers to the ability for the full text search to pick up on common misspellings or word stems and apply them to the search results. 

Sentiment Analysis

As part of both Trending Topics powered by Bright™ and the results of full text search, we’ll provide an aggregate view of the sentiment the topic or the search term(s). WorkStep uses Google’s natural language processor (NLP) to determine the sentiment of each open text comment and does not calculate sentiment for comments that are 1 word or less. Signatures do not count toward the word count of comments. 

Sentiment Analysis Logic

Sentiment is bucketed into 3 groups: negative, positive or neutral. If the NLP has low confidence in the sentiment bucket we will consider it neutral sentiment. If a sentiment cannot be determined, it will be considered “null” and will not be included in the sentiment graph or counts.

Reporting

Clicking into a Trending Topic or the results of a full text search will generate a list of comments that make up that topic/search, an overall frequency graph, an overall sentiment graph, frequency by attribute graph, and sentiment by attribute graph.

Calculations

The frequency graph is calculated by the individual comments and begins on Sunday of each week. As an example, if an employee mentioned “schedule” 4 times in a single check-in and a user searched for “schedule,” we would count that 4 times in the frequency chart for those full text search results.

Attribute charts will respect the 3 or more anonymity threshold. This means that if there is a comment that comes from a Forklift Operator and that’s the only Forklift Operator in those search results, we will not show a bar for that role on the by role attribute charts. 

The sentiment graph hover-over will show the count of individual comments that make up each bucket. Because some comments that are part of the search could have a “null” sentiment, the counts may not always match up to the total comment count.

Comments will display in order from most to least recent and will contain the same chips as on the comments tab - respecting anonymity threshold. You'll be able to export the comments in the same format and including the same fields as the comment export from the comments page by clicking on the download icon in the top right corner of the comment panel:

Term Expansion

Using the same logic and technology used to determine the Topics for Trending Topics by Bright™, we’ll suggest topics for users to add to their search based on their initial search. You can select multiple topics to expand your search, and can continue to add keywords as well. 

Below are examples of how to use the topic search:

• topic:[growth opportunities] – multi word topics must be in brackets
• topic:[innovation] - singe word also works
• topic:[comfort] topic:innovation - multiple topics 
• topic:innovation - single word no bracket works 
• topic:[comfort] feedback - topic search & comment search