Comment Analysis

Updated 8 months ago by Victoria

What is Comment Analysis?

Comment Analysis enables you to easily analyze your open-text feedback at scale. Capabilities include full text search, term expansion, sentiment analysis, and visual reporting of sentiment and frequency.

Report Cards:

By clicking on any row within the Trending Topics table, or searching for a term, users can view a Report card which includes all comments surrounding that topic. The Report card also features Overview, Sentiment, and Frequency charts filterable by state, location, role, age, and gender.

For example: If a user clicks into the “Surveillance” Trending Topic, or searches for that term, the report card will populate with any comments that pertain to the subject within the last 180 days.

Trending Topics

Trending Topics powered by Bright™ is an aspect of Comment Analysis that uses OpenAI’s ChatGPT models to identify topics that are surfacing through open-text feedback. In the platform, it will be referred to as Trending Topics along with a lightbulb icon to indicate this is a Bright™ feature.

To access Trending Topics, navigate to the Feedback dashboard and click on the Analysis tab.

Defining a Topic:

A topic is a grouping of comments discussing the same or similar thing, even though the comments may not mention the topic specifically. A comment can have more than one topic assigned to it. It could also have no topic assigned to it if there isn’t one that is deemed relevant (for example a comment that says, “N/A”).

The list of Trending Topics for a company is the collection of most frequently discussed topics via comments. To qualify as a topic, it needs to be mentioned across at least 8 check-ins in the last 4 weeks across that company and that company needs to have at least 30 check-ins responded to in the last 4 weeks.

Trending Topics powered by Bright™ assigns comments to a topic out of a list of hundreds (~350) of industry specific topics that have been defined through the analysis of historical comment data, manually reviewed, and curated.

Understanding the Trending Topics Table:

The Trending Topics table refreshes daily and includes topic frequency ranking, rank change and sentiment calculated at a company level.

Topic frequency is measured by the unique number of check-ins that mention that topic over the last 4 weeks (28 days).
To calculate rank change, we’re comparing the current 4 week period to a previous 4 week period offset by two weeks (ie: 7/11 - 8/8 compared to 6/23-7/25).
We will show a maximum of 40 topics per company. There is no minimum number of topics that we will show.
If a topic wasn’t previously on the list of topics and is entering the top 40 rank, we’ll count it as 40th rank for the purpose of calculating the rank change.
The default sorting of the Trending Topics table is by frequency rank. Users are able to sort on both the rank change and sentiment columns. Sorting on rank change will sort on the absolute value of the rank change. Users are not able to apply filters to the list of trending topics.

Users will be able to explore company-wide topics that have comments within their scope assigned to them by clicking on the topic in the table. This will bring them to the full text search page with more detailed reporting where they can drill into comments within their scope.

Note: As a rule, we only allow users to see their scope, save for rare circumstances such as viewing company benchmarks, and now Trending Topics.

Trending Topics is an exception to this rule because it is a key way to understand how a user’s scope of the workforce contributes to the broader trends at the company. For example, if overtime is suddenly coming up frequently with negative sentiment, it would be helpful to know that your building actually has relatively better (or worse) sentiment than the org on average, or perhaps isn't really being mentioned.

Managing Trending Topics:

All users at all companies will be given access to Trending Topics powered by Bright™. Companies can choose to disable Trending Topics while still maintaining access to the other Comment Analysis features. Disabling Trending Topics is done at a company level and cannot be disabled for specific users.

Trending Topics FAQ:

Alert categories are a subset of the topic whitelist, but not all topics are on the alert categories menu. The reasons for this today are:
- Alerts highlight comments that are sensitive or urgent and not all topics fit that description.
- Alter categories are customizable while Topics are not.
Topics are not the same as Opportunities in a few ways:
- Opportunities are based on the quantitative answers to questions, grouped by the question theme while topics are based on the qualitative answer content, grouped by the answer content.
- Opportunities are grouped into more general categories, but can be filtered for a user’s specific scope while topics are more specific categories that will first be presented at a company level.

Full Text Search

Users can input one or many keywords into the search bar to search comments over the last 180 days. When searching multiple keywords at once, separate the words by a space.

Users are able to apply filters to narrow their search. Filters can be applied before they begin their search or after their results have been returned. Please note that all searches will be “OR” searches and the current version does not support exact searches or “AND” searches.Note: An “AND” search would only populate results that include every word included in that search while an “OR” search will populate results that include any of the words included in the search.

Note: Comments displayed via Full Text Search are read-only and are not clickable or able to be responded to from this dashboard. Additionally, users can export the list of comments to an Excel spreadsheet by clicking on the download icon in the top right corner of the comment panel. (See below)

Full-Text Search Language Support:

For comments in other languages, we will search the translation of the comment.

Fuzzy Search

Fuzzy search functionality refers to the ability for the full text search to pick up on common misspellings and word stems and apply them to the search results.

We will not allow the option for users to ignore stemming or common misspellings.
We will indicate the corrected search term that is also being searched for beneath the search bar.
The highlighting within the comment text will include the additional words that were searched for due to stemming.

Sentiment Analysis

As part of both Trending Topics powered by Bright™ and the results of full text search, we’ll provide the aggregate view of the sentiment of either the topic or the search. We use Google’s natural language processor (NLP) to determine the sentiment of each open text comment and do not calculate sentiment for comments that are 1 word or less. Signatures do not count toward the word count of comments.

Sentiment Analysis Logic:

Sentiment is bucketed into 3 groups, either negative, positive or neutral.
If the NLP has low confidence in the sentiment bucket we will consider it neutral sentiment.
If a sentiment cannot be determined, it will be considered “null” and will not be included in the sentiment graph or counts.

Note: In the current version we will not provide sentiment on each individual comment, allow users to filter on sentiment or provide overall sentiment of all comments.

Reporting

Clicking into a Trending Topics OR the results of a full text search will provide the comments that make up that topic/search, an overall frequency graph, an overall sentiment graph, frequency by attribute graph and sentiment by attribute graph.

Calculations:

The frequency graph is calculated by the individual comments and begins on Sunday of each week.
- Example: If an employee mentioned “schedule” 4 times in a single check-in and a user searched for “schedule” we would count that 4 times in the frequency chart for those full text search results.
Attribute charts will respect the 3 or more anonymity threshold.
- This means that if there is a comment that comes from a Forklift Operator and that’s the only Forklift Operator in those search results, we will not show a bar for that role on the by role attribute charts.
The sentiment graph hover over will show the count of individual comments that make up each bucket. Because some comments that are part of the search could have a “null” sentiment the counts may not always match up to the total comment count.

Comments will display in order from most to least recent and will contain the same chips as on the comments tab - respecting anonymity threshold. Users will be able to export the comments in the same format and including the same fields as the comment export from the comments page by clicking on the download icon in the top right corner of the comment panel. (See below)

We will not support exporting of the frequency and sentiment reports for V1.

Term Expansion

Using the same logic and technology we use to determine the Topics for Trending Topics by Bright™, we’ll suggest topics for users to add to their search based on their initial search.

Users can select multiple topics to expand their search and can continue to add keywords as well.

Below are examples of how to use the topic search:

topic:[growth opportunities] – multi word topics must be in brackets
topic:[innovation] - singe word also works
topic:[comfort] topic:innovation - multiple topics
topic:innovation - single word no bracket works
topic:[comfort] feedback - topic search & comment search