The Huddle’s College Survey Data, and My Methods
(No Ratings Yet. Click to Rate!)
Loading ...Quite a lot of commentary to see there.
Yours truly has a long treatise up there; I’d caution against taking a lot of the stuff as gospel though.
My methods for generating those percentages you see was a little different than simply comparing % of answers in each group: to try to control for the different sample sizes (as well as respondent sizes) in each group (whether it’s nationals vs. no-nationals or coaches vs. no-coaches) I aggregated response counts and divided by total number of teams, or players if the question entailed roster counts…essentially creating an average rate for either group, and then looked at differences between these rates. I’d encourage caution to the tune of discounting anything below 10% difference or so as being less significant, but some of the differences are really prominent and that’s the stuff I keyed in on in my opening paragraphs as they’re likely significant.
I worked on this all a few months ago…was channeling my college stats pretty hard but converting the survey responses into something SPSS would be able to work with for rigorous analysis proved beyond my time or pleasure.
All that said, if you want to take a look at the original data set (caution: I didn’t make the sheets with prettiness/usability in mind: probably best reserved for people who enjoy playing with excel) you can download the .xlsx (or .xls if you’re incompatible) here.
UPDATE: I’ve been informed that the data is not for sharing, at the lack of consent of the team captains.You might ask the Huddle for permission if you’re interested in working with the data.
I’d love to see somebody with time, desire, and means hit the data hard(perhaps some of the other Huddle contributors already have and it’s not apparent from their commentary), and I’m sure the Huddle would too.
Very interesting read! Great job!
I too enjoyed reading your take on the data. As far as statistical rigor goes, my take was somewhat similar to yours. I ran the percentages for each question against post-season success at the varying levels. I also did some basic statistical analysis – correlations, standard deviations, and the like. I started out with a very quantitative article, but I moved away from that and decided to present the data in a more qualitative form, trying to focus more on the stories the numbers told than the numbers themselves.
You’re right that there are some methodological challenges here. Not all teams answered all questions, some of the answers were inexact or ambiguous, sample size was somewhat low on the women’s side, and some of the questions relied upon subjective evaluations that were probably applied with different standards (i.e., the captain of a national semifinalist would probably require more of you before giving you a top score for throwing ability than the captain of a sectionals-and-out team would). There are probably others that we didn’t notice.
Still, it’s by far the most organized and comprehensive survey of college ultimate ever done. It was a great idea, and I’m flattered that the Huddle invited me to participate. I’d be really interested to see the results of an identical or similar survey in 5 to 10 years’ time.
Oh yeah, and long time reader, first time caller. You’ve got a pretty cool blog going here.
Pinto,
This return comment is way late–I’m glad to hear you at least got some statistical rigor in there.
I’m really glad to see them do this, too–I had emailed them back around the 50,000 hits mark suggesting they could do some pretty powerful leveraging of the hive mind given how prominent they are, but I never envisioned something as big and cool as this.
Definitely be interesting to see the trends; here’s hoping it keeps up.
By the way, I’m astounded that anybody can routinely wade through the RSD much, much less attempting to clean it up. Hang in there.