Comment Analysis
Updated
by
Victoria
Our Comment Analysis feature enables you to easily analyze your open-text feedback at scale. Capabilities include full text search, term expansion, sentiment analysis, trending topics, and visual reporting of sentiment and frequency.
Trending Topics
Trending Topics powered by Bright™ is a component of the Comment Analysis feature that uses OpenAI’s ChatGPT models to identify topics that are surfacing through open-text feedback. In the platform, it will be referred to as Trending Topics along with a lightbulb icon to indicate this is a Bright™ feature.
To access Trending Topics, navigate to the Feedback dashboard and click on the Analysis tab.
Defining a Topic:
A topic is a grouping of comments discussing the same or similar thing, even though the comments may not mention the topic specifically. A comment can have more than one topic assigned to it. It could also have no topic assigned to it if there isn’t one that is deemed relevant (for example a comment that says, “N/A”).
The list of Trending Topics for a company is the collection of most frequently discussed topics via comments. To qualify as a topic, it needs to be mentioned across at least 8 check-ins in the last 4 weeks across that company and that company needs to have at least 30 check-ins responded to in the last 4 weeks.
Trending Topics powered by Bright™ assigns comments to a topic out of a list of hundreds (~350) of industry specific topics that have been defined through the analysis of historical comment data, manually reviewed, and curated.
Understanding the Trending Topics Table:
The Trending Topics table refreshes daily and includes topic frequency ranking, rank change and sentiment calculated at a company level.
- Topic frequency is measured by the unique number of check-ins that mention that topic over the last 4 weeks (28 days).
- Rank change is calculated by comparing the current 4 week period to a previous 4 week period offset by two weeks (ie: July 11 - Aug 8 compared to June 23 - July 25).
- While the platform will show a maximum of 40 topics per company, there is no minimum number of topics that must exist before topics are presented.
- If a topic wasn’t previously on the list of topics, and is entering the top 40 rank, it will be considered ranked as 40th for the purpose of calculating the rank change.
- The default sorting of the Trending Topics table is by frequency rank. You'lld be able to sort on both the rank change and sentiment columns. Sorting on rank change will sort on the absolute value of the rank change.
You'll have the option to explore comments within your scope that are related to company-wide topics by clicking on the topic from within the table. This will bring you to the full text search page with more detailed reporting, where you can drill into relevant comments.
Managing Trending Topics:
All users will be given access to Trending Topics powered by Bright™. Employers can choose to disable Trending Topics while still maintaining access to the other Comment Analysis features. Disabling Trending Topics is done at a company level and cannot be disabled for specific users.
Trending Topics vs. Categories and Opportunities:
- Alert categories are a subset of the topic whitelist, but not all topics are on the alert categories menu. The reasons for this today are:
- Alerts highlight comments that are sensitive or urgent and not all topics fit that description.
- Alter categories are customizable while Topics are not.
- Topics are not the same as Opportunities in a few ways:
- Opportunities are based on the quantitative answers to questions, grouped by the question theme while topics are based on the qualitative answer content, grouped by the answer content.
- Opportunities are grouped into more general categories, but can be filtered for a user’s specific scope while topics are more specific categories that will first be presented at a company level.
Report Cards:
By clicking on any row within the Trending Topics table, or searching for a term, you'll be able to view a Report card which includes all comments surrounding that topic. The Report card also features Overview, Sentiment, and Frequency charts filterable by state, location, role, age, and gender.
As an example, clicking into "Surveillance" from the Trending topics table or searching the term "Surveillance" will generate a Report Card containing all comments that pertain to Surveillance in the last 180 days.
Full Text Search
Looking for comments related to a particular topic? Input one or many keywords into the search bar to search comments over the last 180 days. Tip: When searching multiple keywords at once, separate the words by a space.
You'll be able to apply filters to narrow results either before you begin searching or after results have been returned. Note that all searches will be “OR” searches and the current version does not support exact searches or “AND” searches. An “AND” search would only populate results that include every word included in that search, while an “OR” search will populate results that include any of the words included in the search. For comments in languages other than English, the platform will search the translation of the comment.
Fuzzy Search
Fuzzy search functionality refers to the ability for the full text search to pick up on common misspellings or word stems and apply them to the search results.
Sentiment Analysis
As part of both Trending Topics powered by Bright™ and the results of full text search, we’ll provide an aggregate view of the sentiment of either the topic or the search. WorkStep uses Google’s natural language processor (NLP) to determine the sentiment of each open text comment and do not calculate sentiment for comments that are 1 word or less. Signatures will not count toward the word count of comments.
Sentiment Analysis Logic:
Sentiment is bucketed into 3 groups, either negative, positive or neutral. If the NLP has low confidence in the sentiment bucket we will consider it neutral sentiment. If a sentiment cannot be determined, it will be considered “null” and will not be included in the sentiment graph or counts.
Reporting
Clicking into a Trending Topics or the results of a full text search will generate a list of comments that make up that topic/search, an overall frequency graph, an overall sentiment graph, frequency by attribute graph and sentiment by attribute graph.
Calculations:
The frequency graph is calculated by the individual comments and begins on Sunday of each week. As an example, if an employee mentioned “schedule” 4 times in a single check-in and a user searched for “schedule” we would count that 4 times in the frequency chart for those full text search results.
Attribute charts will respect the 3 or more anonymity threshold. This means that if there is a comment that comes from a Forklift Operator and that’s the only Forklift Operator in those search results, we will not show a bar for that role on the by role attribute charts.
The sentiment graph hover over will show the count of individual comments that make up each bucket. Because some comments that are part of the search could have a “null” sentiment the counts may not always match up to the total comment count.
Comments will display in order from most to least recent and will contain the same chips as on the comments tab - respecting anonymity threshold. You'll be able to export the comments in the same format and including the same fields as the comment export from the comments page by clicking on the download icon in the top right corner of the comment panel:
Term Expansion
Using the same logic and technology used to determine the Topics for Trending Topics by Bright™, we’ll suggest topics for users to add to their search based on their initial search. You can select multiple topics to expand your search, and can continue to add keywords as well.
Below are examples of how to use the topic search:
- topic:[growth opportunities] – multi word topics must be in brackets
- topic:[innovation] - singe word also works
- topic:[comfort] topic:innovation - multiple topics
- topic:innovation - single word no bracket works
- topic:[comfort] feedback - topic search & comment search
