Scholarships/Survey

Please find below (and feel free to edit too) some highlights of the results of a survey that aimed to help evaluate the Wikimania Scholarship Program by gathering anonymous feedback from its candidates and awarded recipients in 2010. I'm also sharing the links to the entire table of responses, with the aim of encouraging more people to join in and further analyse the remaining data. TSB 16:33, 30 September 2010 (UTC)[reply]

Background

Out of nearly 2,500 individuals that applied for a scholarship in 2010, only 3% were awarded. Approximately 70% were screened out in the first round of evaluation because their responses did not fit the selection criteria. Although the remaining 30% had potential to be awarded with scholarships, only 72 were selected after a deep and thorough second round evaluation. The approximate US $110,000.00 budget allocated by the Wikimedia Foundation to the Scholarship Program was respected. The awarded candidates travelled to Gdansk and attended the conference, but a considerable number of candidates did not receive a scholarship and unfortunately could not attend the event.

Survey

One online survey with 5 exploratory and 22 likert scale questions was simultaneously sent to two groups of respondents between September 20 and 26, 2010. Questions were skippable and participation was anonymous and voluntary.

One surveyed group is called CANDIDATES. It represents those not awarded with a scholarship in 2010, but with enough potential to have received one. Fifty-six individuals, or the equivalent of approximately 8% of nearly 680 non-awarded candidates that reached the second round evaluation, completed the survey and shared their anonymous responses.

The other surveyed group is called RECIPIENTS. It represents participants that were awarded with a scholarship to attend Wikimania in 2010. Forty-two individuals, or the equivalent of approximately 60% of all awarded recipients, completed the survey and shared their anonymous responses.

Results

Scholarship Program

Candidates gave a 3.52 score (out of a 5 point scale) to the “Overall” evaluation of the Scholarship Program, while Recipients gave a 4.24 score. The Scholarship Program was considered “good” or “very good” for 39% of Candidates and 88% of Recipients.

The difference between the scores given by the two groups is significant in all categories. The three categories that represent the biggest gap between the evaluation of Candidates and Recipients (“Selection Criteria”, “Scholarship Process” and “Contact with Candidates”) are also the categories that obtained the lowest scores among both groups of respondents. Between 30%-45% of Candidates evaluated these three categories coordinated by the Scholarship Team as “poor” or “very poor”, as illustrated by the comments below:

“The selection process was very mysterious. There was no contact with the applicants beyond an approval/rejection letter, and the actual criteria were not very clear at all.”
“this years' process and selection was absolutely poor and shady.”
“I don't know what went down in the selection process”
“The selection criteria was insufficiently explained.”
“I don't know what kind of geographical criteria was followed”
“many valid scholarship applications were ruled out while a lot of people without any special commitment or involvement with Wikimedia projects did get a scholarship.”
“out of 2500 applications 70 is basically nothing.”
“many of the communications were too late to the point where some people actually did not come.”
“the whole procedure should be revised in order to increase both transparency and fairness of results”

On the other hand, the categories “Call for Application” and “Application Form” were positively evaluated by both groups of respondents (“good” or “very good” for nearly 60% of Candidates and for above 70% of Recipients). Additionally, Recipients gave a 4.33 score to the category “Travel arrangements”, with 78% of respondents considering it “good” or “very good”, as illustrated by the testimonials below:

“travel arrangements were fantastic -- superb, efficient, and quick service.”
“I arrange the travel with the agency in the most comfortable and pleasant way (they have been really kind, precise and professional)”

Wikimania Event

Recipients gave a 4.27 score to the “Overall” evaluation of the Wikimania Conference, with 76% of respondents considering it “good” or “very good”, as illustrated by the testimonials below:

“Very constructive and useful for the quality of the contents of the talks and posters”
“Nice venue and conference overall”
“Really good conference, well organized considering the dimensions of the event.”
“Good talks and good for meeting very interesting people.”

Most categories reached mean scores of 4.0 or higher, but “Visa Arrangements” and “Hotel Accommodation” were given lower scores (3.44 and 2.69, respectively). Almost half of Recipients (48%) considered “Hotel Accommodation” as “poor” or “very poor”.

“hotel arrangements were not efficient”
“the situation with housing was totally and completely unacceptable.”
“Many people were quite upset because of the inconvenience (accommodation)”

Self-evaluated Contribution

Both groups showed an increase in their self-evaluated contribution to the Wikimedia Movement from “before applying for a scholarship” to “after applying for a scholarship”. The percentage of self-evaluated “high” or “very high” contributors slightly increased (3 percentage points) among Candidates, from 40% to 43% “after applying for a scholarship”. The percentage of “high” or “very high” contributors among Recipients, on the other hand, strongly increased (18pp) from 58% to 76% after receiving a scholarship, as illustrated by the testimonials below:

“I'm much more active and engaged in the movement -- which I didn't expect! Since then, I have dipped my toe into participation with other projects and with a forming chapter.”
“my level of awareness, participation, and contribution to the overall Wikimedia movement is definitely increased and I feel much more involved in the global and international movement.”
“After attending Wikimania, I'm more enthusiastic about the power and potential of free culture, and want to attend future Wikimanias!”
“The ability to interact with other Wikimedians definitely impacted my research and helped me understand a few things about the community.”
“It has inspired me to enhance my level of contributions for Wikimedia.”

Impact on Interest

For 93% of Recipients, the Scholarship Program had a “high” or “very high” impact on their interest to participate in the Wikimania Conference. Many Recipients shared testimonials that illustrate this impact, as follows:

“I definitely would not have been able to participate without the scholarship program.”
“Without the scholarship this wouldn’t have been possible!!!”
“it would have been impossible to attend the Wikimania.”
“It was great to see the very diverse group of people from around the world that the scholarship program made possible.”

Percentage-wise, twice as many Recipients as Candidates considered that the Scholarship Program had a “very high” impact on their interest to participate in the “Wikimedia Movement”, “Wikimania Conference” and “Scholarship Program”. Nonetheless, nearly 20% of Candidates still attributed to the Scholarship Program a “very high” impact on their interest and participation.

Comments and Suggestions

“start the process earlier”
“(provide) clear information about whether would there be special scholarships for speakers”
“more people could have been allowed to attend with partial scholarships”
“inform applicants of other scholarship programs, eg the Wikimedia France”
“I was very surprised by the gender imbalance at the conference; I would like to see follow-up research done about the actual statistics”
“Please consider including scholarship support for Wikisym ”
“(provide) feedback from those who attended the conference”
“Improving transparency during the selection process would be the single greatest improvement”

Questions and Responses