Internal - Faculty & Staff

Student Experience

In FY’18, Simmons partnered with Gallup to analyze student satisfaction data collected from online masters students in 2U enabled programs, to help inform retention efforts. This research has incorporated both quantitative and qualitative research and involves surveying our graduate and undergraduate student populations with questions pertaining to their satisfaction, emotional attachment and likelihood to recommend Simmons. The program has since extended to include campus graduate students in FY’19, and to undergraduate students in FY’20.  

Key Metrics

Net Promoter Score The primary metric Simmons uses to measure the student experience. NPS is based on a single question: “How likely are you to recommend X to your friends or colleagues”. Simmons tracks students’ likelihood to recommend their program, and their likelihood to recommend the university. Gallup’s research for Simmons’ online graduate programs showed that the NPS question predicted significantly higher probabilities of retention.
Satisfaction is measuring on a 1-5 scale how well Simmons is meeting the expectations for students who are paying to attend the university.
Emotional attachment is a proprietary measure that assesses the level of enthusiasm and engagement that students feel towards Simmons.

Graphic with animated people broken into three groups: detractors, passives, promoters.

Methodology

Data Collection All survey data is collected via a web-based survey, administered by Gallup each term. Program feedback is collected from all enrolled students, and in the case of graduate students, for each course they are enrolled in.  Undergraduate student program data is collected once per year in the fall semester.  Click here for a full schedule of survey dates.

Chart with terms and survey dates

Analysis and Reporting Gallup applied a regression model to determine the importance of a series of drivers to outcomes of interest to Simmons including Simmons University NPS and Program NPS.  See a high-level methodology overview.

Repeated Measures
To account for repeated measures (students that take the survey multiple times over the course of their program) Gallup looks at a student’s most recent response to determine NPS.  All data that is shared with Simmons is de-identified, and completely anonymous.

Deliverables

There are many deliverables available to Deans and program managers throughout the academic year.

Research Highlights Report: Summary report including key metric trends, drivers of NPS, other noteworthy findings and retention for individual graduate programs. Delivered twice annually, in April and October.
Dynamic Scorecard: Interactive online data report with trend data on NPS and retention. Multiple views by university, college, and program levels. Updated three times annually for all graduate programs.
Faculty Scorecard: Faculty-level reports containing detailed student feedback about their experience with the course and faculty members. Scorecards containing data from each term will be available for all online programs approximately two weeks following the end of that term.
Crosstabs: Frequency of responses for all survey questions organized by key categories, e.g., demographics. Updated three times annually for Nursing, Social Work, and Library Science.
Net Benefit Analysis: Analysis conducted to measure program-level increases in revenue based on incremental improvements in retention. For programs with sufficient sample size, this also includes a tool that allows program leadership to calculate potential revenue increases based on improvements in discrete survey items.

See a full list of report deliverables.

Messages from the Field

“The data Gallup has collected and analyzed has been critical to informing curricular and programmatic changes in the [email protected] program. It has enabled us to systematically use student feedback to improve their experience in the program. Both the qualitative and quantitative data collected from every student, every course, every term assures we are providing students what they need to optimally learn and thrive in the [email protected] program. 

Our students are the experts in evaluating their individual and collective experiences and utilizing the outcomes data will guide teaching and learning and provide students with the confidence that they are heard and valued — supporting a strong student centered culture. The positive feedback students provide regarding their faculty and course work is extremely uplifting to our faculty and helps faculty to build on what is working.  Negative feedback also provides value in that we can use it as reflection and to coach faculty around areas of opportunity for improvement,   Constructive feedback also enables curricular insights that might be changed to help future students learn more effectively. 

The data driven approach that STEP brings to [email protected] is helping to assure we maintain a stellar academic program! “

Heather Schlosser, Associate Dean School of Nursing

Survey Materials

Questionnaires are designed to gather feedback on students’ experiences in their program and in the case of graduate students, for each course they are enrolled in. Graduate surveys were initially designed by Simmons faculty for use in the 2U enabled programs, and include the addition of a handful of NPS questions to compare across partner programs.  The Undergraduate survey was created working closely with the Dean and Gallup.  Copies of each survey can be found below:

Graduate Course questionnaire
Graduate Program questionnaire
Undergraduate questionnaire

Results

Simmons measures NPS on both a program and a university level.  Net Promoter Scores at Simmons vary widely by degree program, but overall results by academic level are displayed below.

Academic LevelUniversity NPSProgram NPS
Graduate 2429
Undergraduate-25-9

Net Promoter Scores vary by industry, and higher education institutions are increasingly relying on student experience metrics to drive strategy.  For comparison data, and an overview to NPS by industry please see this year’s Net Promoter Benchmarks from NICE Satmetrix.

See the Net Promoter Benchmarks.

Frequently Asked Questions

What is STEP?

The STudent Experience Program is a commitment and approach to more actively and systematically use feedback in the academic and administrative management of the university to improve and enhance the student experience and faculty and staff engagement.  It involves surveying our graduate and undergraduate student populations with questions that include demographics, student satisfaction and emotional attachment to Simmons.  STEP also involves administering an engagement survey to faculty and staff.

Why are we doing this?

The Simmons Board of Trustees has charged the organization to implement a more rigorous form of measuring and tracking the student experience. Our current methods of getting feedback from students vary by College and program.  With more consistent and complete data, we’ll be able to address retention challenges in a more targeted way and provide insights helpful to the faculty, administration,  and helpful to action planning.

Who is leading this initiative?

This is a Board-driven initiative, sponsored by The Office of the President and led by Suzie Murphy, VP of Strategic Initiatives.   Associate Provost and Dean of the Undergraduate Program, Catherine Paden is leading the workstream focused on the UG student segment.  Dean Stephanie Berzin and Program Director Heather Shlosser have been leading the workstream focused on the online graduate student work to date as we started with the online student segments in [email protected] and [email protected] programs, building upon the foundation 2U had helped architect at  launch. Various offices supporting the work have included Institutional Research, Strategic Initiatives, Office of Institutional Advancement, Finance and the Provost’s office.

Who is administering the surveys?

Gallup is administering the surveys via email. Gallup is an internationally recognized expert market research analytic company with one of the most highly regarded higher education vertical in the country.  Gallup uses many measurements and benchmarks, and for our programs is using a measurement called the Net Promoter Score or NPS for short.

Why are we using NPS as a metric?

The Net Promoter Score was developed as a business tool to measure the likelihood of customers recommending products or services. It is a reliable and valid metric for customer satisfaction and engagement that predicts enthusiastic use of the company’s products or services.

Do other higher ed institutions use this tool?

Yes. Higher education institutions are increasingly relying on student experience metrics to guide strategy.

How useful is NPS in HE as it does not measure student outcomes?

NPS has been used mostly outside of higher education but there is good reason to believe that it is just as valid within the higher education sector. Extensive Gallup survey research shows that alumni who are more satisfied and engaged were more likely to have graduated on time and were more likely to donate to their alma matter. Across institutions, schools with higher recommendation scores from their alumni consistently have higher retention and graduation rates.

Gallup’s research for Simmons’ online graduate programs showed that the NPS question predicted significantly higher probabilities of retention across students, even controlling for their demographic characteristics.

Is the data confidential?

Yes, the data is confidential.

Who will see the data?

There are several reports that will be generated from the surveys.  The Board will receive an executive dashboard report at every Board meeting with trends in NPS and drivers of NPS.  Deans from each College will receive a separate and more granular report containing individual course and faculty data.  The President, Provost and Deans only will have access to this data. The data will be managed by a “data custodian” and governed by a policy pertaining to access to the data and additional analysis.

How will this data be used?

This data will be used to inform unique retention initiatives across student segments.

What other measures is Simmons collecting to highlight the student experience?

The end of term surveys currently include various measures of student engagement including emotional attachment to Simmons, the quality of their relationships with their professors and fellow students, and questions designed to identify whether they are at risk of leaving the program before completing their degree.

Why are we using NPS over one of these other metrics?

Gallup will be performing a detailed analysis of all the survey items that are being collected and their relationship to retention. The focus has been on NPS (and will likely remain on NPS) because it has been validated across many industries and thus has performance benchmarks against which Simmons can compare itself.

What kinds of questions feed the NPS score?

NPS scores are based on the following question: “How likely would you be to recommend [Simmons University] to friends or family?”

What is “emotional attachment” measuring?

Emotional attachment is measuring the level of enthusiasm and engagement that students feel towards Simmons, its faculty, and their fellow students.

What is “satisfaction” measuring?

Satisfaction is measuring how well Simmons University is meeting the expectations of those who are paying to attend the university.