Do your agency's programs deserve an "A"?

Participant evaluation questionnaires are one way
to get feedback about the popularity and the
quality of recreational activities.

By Gary F. Balling

One challenge recreation professionals always face is to successfully devise new ideas. They must also have the ability to adapt, and change organizations and programs to fit the changing needs of the public.

Americans increasingly spend more money pursuing new ways to enjoy their leisure hours. In order to compete for a share of the consumer's leisure dollar, recreation professionals must constantly monitor the participants' satisfaction with the recreational activity.

Recreation agencies now realize that new and improved programs hold the key to their survival and growth. It is not unusual for program fees to produce upward of 60 percent of a department's budget, with the other 40 percent coming from taxes and miscellaneous sources of revenue.

Recreation professionals are continuously looking for ways to increase the quality and the standards of their services. This is especially true when someone challenges them on the issues of user fees and accountability.

Need for accountability

Many professionals play the numbers game or use "supply and demand" as their basis of evaluation. They measure success by using criteria oriented toward quantity. They assume that a program should be continued if it has a large number of individuals registered or participating. However, using quantity as the only measure does not allow program quality to be sufficiently reviewed.

Many times recreation professionals become so preoccupied with the day-to-day task of maintaining established services, that they give little attention to evaluating the quality of those programs. The demands and limited resources of the recreation organization can also make it difficult for recreation professionals to keep pace with the duties of the job.

Priority must be given to the importance of program evaluations so the organization can adapt and change to fit the changing wants of its clientele. One way to assess participant satisfaction is through written questionnaires.

Survey results

In January 1987, 265 surveys were distributed to superintendents (administrators) of recreation departments at Illinois park districts, and city and county recreation departments. There were 183 respondents, for a return rate of 69 percent.

The survey sought to determine:

• the extent to which participant evaluation questionnaires are being used and the reasons for their use.

• if funds are available to implement the use, development and training of personnel who will

Illinois Parks and Recreation 23 March/April 1988

Participant questionnaires

• administer and interpret the questionnaires.

• when the questionnaires are being used.

Results

The survey revealed that 67 percent of the agencies are using questionnaires to assess participant satisfaction; 33 percent of the agencies do not use questionnaires. In addition, the survey showed that 94 percent of the responding agencies do not budget or identify funds specifically for administering program evaluation questionnaires.

Also, 92 percent of the respondents indicated that no training is provided for the staff who are administering and interpreting program evaluation questionnaires. Ninety-two percent of the respondents indicated that the questionnaires used by their agency were developed in-house by full-or by part-time staff.

The majority (87 percent) indicated they most frequently use program evaluation questionnaires at the conclusion of a program session. Fifty percent of the responding agencies distribute program evaluation questionnaires after each program session. Twenty-eight percent distribute them once a year, and 22 percent distribute them randomly.

Recommendations

Based on the survey response, these general recommendations for

Illinois Parks and Recreation 24 March/April 1988

park and recreation agencies are offered:

1. Park and recreation agencies still need a viable, cost-effective program evaluation questionnaire model to determine participant satisfaction. When this model is designed for use by recreation professionals, it must take into consideration the cost of the evaluation process.

Full cost recovery should include design, training, distribution, interpretation of results, collection and publication. By identifying costs, recreation professionals can determine if the evaluation is viable for their agency, and allocate funds for implementing the process. When reviewing the available current models, none take the implementation cost into consideration.

2. There is a need for specific training in the use of program evaluation questionnaires. The majority of program evaluation questionnaires are developed by staff. However, there are relatively few agencies which provide training in the development and use of evaluation questionnaires.

3. The use of program evaluation questionnaires must be linked to the goals of the organization. Only then can the questionnaire be viewed as a priority and subsequently reinforce the organization's goals.

The credibility of the agency can be strengthened through the use of valid and consistent evaluation methods, and

Illinois Parks and Recreation 25 March/April 1988

Participant questionnaires

the subsequent reporting of evaluation results in annual reports or in other documents.

Observations

Respondents were requested to enclose a copy of their agency's current evaluation questionnaire with the completed survey.

• As a general observation, only one of the 100 questionnaires returned was professionally typeset.

• Most questionnaires were no longer than one page.

• The printing on many questionnaires was blurred and distorted.

Incentives encouraging completion of participant questionnaires were used by 23 percent of those responding. The majority of those use stamped return envelopes; others use discounts on future registrations or special prize drawings to encourage people to return questionnaires.

An appealing, professionally looking questionnaire and return incentives are important components of a successful evaluation process and an overall marketing effort. The bottom line is that recreation professionals are concerned about insuring satisfaction and looking for repeat business.

Learning Resources network cites Nancy K. Austin, co-author of Passion for Excellence, in a 1986 publication to reinforce the importance of evaluation. Austin uses the term learners; those involved in providing recreational services might use the term participants or people.

Here are a few things Austin says:

We should over-evaluate. We should ask our learners for their feedback, their response, their sug-

Illinois Parks and Recreation 26 March/April 1988

We should measure learner satisfaction. Course evaluations are just one way to get learner feedback, and even then, we have no overall measure for learner satisfaction. We should set up a way to measure learner satisfaction, such as a percentage of learners who say they are satisfied overall with our courses. Then we should publicize it, record it, compare it to last year and project it for next year.

The use of program evaluation questionnaires is important when assessing the quality of programs, but it is only one way to assure that participants are getting what they expected. Recreation professionals also need to evaluate by paying a personal visit to programs, talking with the people who participate and watching their reactions.

A better understanding of what people like and dislike will keep them returning to our activities and facilities smiling, excited and prepared for more fun.

ABOUT THE AUTHOR: Gary F. Balling is the administrative manager of the Morton Grove Park District and the Recreation Programming Section representative on the Illinois Park and Recreation Association's board of directors. He completed his master's degree in the management and development of human resources at the National College of Education. His thesis topic was the Assessment of the Use of Program Evaluation Questionnaires by Recreation Professionals to Determine Participant Satisfaction.

References and composite survey results for material used in this article are available from the author.

Illinois Parks and Recreation 27 March/April 1988