Crossword SNITCH

A quantitative guide to the degree of difficulty of the Times Cryptic Crossword

Solver Types and a Neutrino Detector

The SNITCH has always relied on a set of Reference solvers to calculate the difficulty of the crossword. Updating this list over time has led me to look at different types of solver. In particular, it has been important to detect and remove "Neutrinos" from the data (see further below). Once this is done we can look at other types of solver more clearly.

What's a Neutrino?

Neutrino – A person who manually solves the puzzle on paper, and then goes to the Crossword Club and types in the answers as rapidly as possible, either in order to give a high position on the leaderboard or because they are entering a typing competition. (Times for the Times glossary definition)

Have you noticed that the top of the Times Cryptic Crossword Club leaderboard is occupied by solvers who have some remarkable completion times? How does someone solve the puzzle in around 2 minutes, when it probably takes that long to just write in the answers?

On the Times for the Times blog site, we've come to know these solvers as "Neutrinos", a term coined by Tony Sever some years back to describe those people who like to occupy the top spots on the leaderboard without actually being top-notch solvers. The theory is that these people do the crossword on paper, then transcribe the answers on the Crossword Club interface.

And the big question is: how can we detect these? Some of them are pretty obvious: if you look at the results at the table on the right, the top solver in recent months is sudokulover with an average solving time of 02:16 (min:sec). Given that Mark Goodliffe (10+ times winner of The Times Crossword Championship) has an average solving time of around 6 minutes, we're very sure that this is not a real solving time. But it's much harder to tell when you're talking about average times in the range of 5 to 8 minutes. Who are the Neutrinos and who are the real solvers? This page tries to answer that question.

Clusters of solvers

The chart below shows a way to separate Neutrinos from other solvers. It plots each solver's average solving time (in seconds) on the x-axis against an "alignment to SNITCH" factor on the y-axis. This alignment factor is 1 when, on average, a solver's personal NITCH rating for a crossword is proportional to the SNITCH rating for that crossword. It is 0 when there is no alignment. (For those interested, more detail on this is provided below).

The blue dots on this chart represent the Reference solvers for the SNITCH as of October 2019. (Note that you can hover over each point to see the name of the individual solver.) As you'd expect, these are clustered around an alignment factor of 1 (generally between 0.5 and 1.5) regardless of their average solving time.

The red dots represent solvers who have been around for some time but are not Reference solvers. A group of these is spread across the range of average solving times and also spread about the SNITCH alignment factor of 1. But there's a cluster of red dots in the lower left corner that has low solving times and is spread around an alignment factor of 0. These are the Neutrinos.

This data suggests that any solver with a solving time of less than, say, 8 minutes (480 seconds) and a SNITCH alignment factor of, say, less than 0.5 could be safely classified as a Neutrino.

The grey dots represent new solvers since the Reference solver list was updated early in 2019. We can now see that some of these appear to be Neutrinos (Blue Moon, for example), while others - even with low solving times - appear to be real solvers (e.g. Lihar). It would be very difficult to make this assessment without checking the alignment with the SNITCH, so it clears up any speculation that might arise for these.

Of course, this analysis is not 100% guaranteed, but it looks like a good start.

Remaining Questions

It's curious to see that a couple of solvers (Paul Gilbert and Mike Harper) are "super-aligned" to the SNITCH, i.e. they seems to find harder puzzles especially hard. From looking at the underlying data, my guess is that this is due to a relatively small sample size with a few results pulling the alignment higher.

It is a bit hard to know what's going on with the low-alignment, higher solving average results. There are quite a few with an alignment of less than 0.4, including a handful of Reference solvers. These are presumably not Neutrinos in the Club definition sense (unless they are just very slow typists). But if they are genuine solvers, we'd probably expect them to align to the SNITCH more closely. Maybe there is some underlying inconsistency (e.g. there are actually several solvers of unequal ability doing different crosswords or the solver sometimes has help and sometimes doesn't). These are speculations with no real way to test them.

Updating solver status

This analysis allows me to reclassify solvers, as shown in the chart below. One of my main motivations is to be able to update the list of Reference solvers reliably. I'd like to be able to include new solvers (especially good ones) without accidentally including Neutrinos. And I'd like to track other non-Reference and non-Neutrino solvers, even if their data isn't used for the SNITCH.

So I can now classify solvers as follows:

Neutrino
Those with fast solving times and low alignment to the SNITCH (shown in orange)
Reference Solvers
Those aligned to the SNITCH and having an average solving time of less than 45 mins (shown in blue)
Tracked Solvers
Those that don't meet the conditions for Reference Solvers but I can be fairly sure are not Neutrinos (shown in red)

New solvers will come into the list over time and can be reclassified on my next update.

I've shown the old and new status in the full table to the right so that you can track the changes for each user.

So what's the SNITCH alignment factor?

This is actually pretty straightforward if you know a little maths. I plot the solver's individual NITCH scores agains the corresponding SNITCH for the same puzzle and measure the slope of the line of best fit. A slope of 1 indicates that the results are proportional, a slope of 0 means that there's no correlation.

For example, verlaine has results as shown in the chart below. Easier puzzles (according to the SNITCH) are on the left of the x-axis, harder ones on the right. Verlaine's NITCH results vary, but are generally higher as we progress from left to right. The overall slope of the line of best fit is 0.95, showing good alignment with the SNITCH on average.

In contrast, sudokulover has results as shown in the chart below. There is no correlation with the hardness of the puzzle according to the SNITCH. In fact, the slope is negative, meaning that on average the harder puzzles are easier for sudokulover than the easier ones. This lack of correlation is exactly what you'd expect if the solver is just typing in known answers.

Solving times for club members

The table below shows crossword club solvers who have submitted more than 20 solutions in the last six months, with an average solving time of less than 60 minutes for an error-free crossword. They are ranked by that solving time. It also indicates their status, including whether they are Reference solvers for calculation of the daily SNITCH.

Rank Solver Status New Status Average Time SNITCH Alignment
1 patricia_briggs new_solver neutrino 02:00 -0.35
2 2 Down tracked neutrino 02:05 0.14
3 sudokulover tracked neutrino 02:16 -0.05
4 Trevor Cooke tracked neutrino 02:22 0.23
5 Sujeri tracked neutrino 02:49 0.08
6 TimA tracked neutrino 02:57 0.08
7 Margaut tracked neutrino 02:59 0.03
8 William Corder tracked neutrino 03:01 0.11
9 ian.rann tracked neutrino 03:08 -0.03
10 FunnyBunny tracked neutrino 03:19 -0.12
11 Westcoaster tracked neutrino 03:20 -0.09
12 Nigel1953 tracked neutrino 03:24 0.19
13 davidcrow tracked neutrino 03:35 0.05
14 GolfDocAndy tracked neutrino 03:37 0.01
15 BrianS tracked neutrino 03:37 0.03
16 juntaK tracked neutrino 03:37 0.04
17 Blue Moon new_solver neutrino 03:50 -0.04
18 Cryptic2015 tracked neutrino 03:54 0.23
19 adriano tracked neutrino 03:54 -0.18
20 99evers tracked neutrino 03:54 0.2
21 Gwyrfai tracked neutrino 03:55 -0.05
22 Dunsolvin new_solver neutrino 04:03 0.18
23 leo tracked neutrino 04:14 0.07
24 Henry Wyatt tracked neutrino 04:16 0.05
25 edwards.robert1 tracked neutrino 04:20 0.01
26 ourjohnny new_solver neutrino 04:23 0.37
27 Gunnergray tracked neutrino 04:24 -0.08
28 rjr520 tracked neutrino 04:24 0.18
29 Rsedge new_solver neutrino 04:31 0.07
30 Mikemon tracked neutrino 04:44 0.29
31 ray lewis tracked neutrino 04:47 -0.0
32 kalakuta69 tracked neutrino 04:52 -0.24
33 greggybell tracked neutrino 05:06 0.23
34 Sye tracked neutrino 05:08 0.1
35 mohn reference reference 05:20 1.01
36 ajb9 new_solver neutrino 05:33 -0.07
37 chaswil tracked neutrino 05:49 0.07
38 Magoo reference reference 06:10 0.89
39 robertsonjon new_solver tracked 06:24 1.3
40 Splottite new_solver neutrino 06:35 0.18
41 tiljo tracked neutrino 06:39 0.04
42 Pontius tracked neutrino 06:46 0.08
43 Eamon Ryan reference reference 07:05 1.03
44 CMB new_solver neutrino 07:27 -0.29
45 Jason reference reference 07:33 1.13
46 Avid Crossworder new_solver neutrino 07:42 -0.02
47 verlaine reference reference 07:51 0.91
48 mistigris reference reference 08:40 0.97
49 aphis99 reference reference 08:45 1.08
50 Lihar new_solver reference 08:48 0.67
51 Midas reference reference 09:30 0.76
52 AlanS reference reference 09:45 1.02
53 boulemiche tracked reference 09:47 0.77
54 hilarym tracked reference 09:56 0.94
55 richard reference reference 10:01 1.16
56 asaldanha reference reference 10:04 1.11
57 robertL tracked reference 10:13 0.69
58 chindit reference reference 10:39 1.07
59 Aloysius tracked tracked 10:40 0.12
60 Derek Harrison reference tracked 10:45 0.24
61 Paul Gilbert tracked tracked 10:51 2.52
62 chriswords reference reference 11:22 0.83
63 glheard reference reference 11:34 0.83
64 ChuckB tracked reference 11:40 0.81
65 tiggis tracked reference 12:02 1.02
66 lickert reference reference 12:13 0.96
67 Tom Stubbs reference reference 12:42 1.09
68 trux reference reference 12:45 1.05
69 Kinghedger tracked reference 12:46 0.82
70 lili tracked reference 13:02 0.94
71 jimpen reference tracked 13:05 0.42
72 Spur58 reference tracked 13:15 0.37
73 noidea reference reference 13:32 0.69
74 fshephe reference reference 13:37 0.62
75 Topical Tim reference reference 13:39 1.34
76 SimonAnthony reference reference 13:46 1.27
77 p.wild reference reference 13:46 0.46
78 neilr reference reference 13:58 1.21
79 John Walsh tracked tracked 14:05 0.28
80 keriothe_ reference reference 14:08 1.05
81 Silva reference reference 14:08 1.05
82 Sheapey reference reference 14:17 1.3
83 Askival tracked reference 14:24 1.05
84 thebarnets new_solver reference 14:25 1.2
85 Heyesey tracked tracked 14:27 1.62
86 mikegrocott reference reference 14:36 0.74
87 andy tracked reference 14:44 1.08
88 Ben Staveley reference reference 14:44 1.0
89 Groucho reference reference 14:50 1.29
90 jim42078 tracked reference 14:50 0.7
91 aphasic tracked reference 14:51 1.31
92 Stavros reference reference 14:58 1.12
93 xyster reference reference 15:08 1.28
94 bigmack68 new_solver reference 15:14 0.98
95 Tibthorpe reference reference 15:17 1.38
96 Big Mick tracked reference 15:17 0.61
97 tunflog tracked tracked 15:25 0.37
98 margaretl tracked reference 15:28 0.87
99 Malcj reference reference 15:39 0.95
100 cyrillic reference reference 15:41 1.03
101 jonathanbaker174 tracked reference 15:47 0.79
102 MikeOsborne reference reference 16:04 0.81
103 HenryBW tracked reference 16:07 1.17
104 Spanna reference reference 16:22 0.54
105 Sue Sweeper reference reference 16:24 1.58
106 David Crooks reference reference 16:38 0.95
107 KenG tracked reference 16:42 0.8
108 rrricharrrd tracked reference 17:14 0.94
109 oshinmark tracked reference 17:22 1.14
110 sotira reference reference 17:44 0.87
111 phthisis reference reference 17:53 0.94
112 RobR reference reference 17:56 1.22
113 Viscomte tracked reference 18:02 0.76
114 EdP tracked tracked 18:05 0.1
115 Fatfinger new_solver reference 18:16 0.9
116 robinh reference reference 18:23 0.87
117 OliviaRhineb reference reference 18:24 1.04
118 JezWoolfrey reference reference 18:30 1.26
119 Carol tracked reference 18:32 1.01
120 fuzz tracked tracked 18:38 0.39
121 wwalker50 new_solver tracked 18:39 1.72
122 MK76 tracked reference 18:50 1.08
123 LouWeed reference reference 19:00 0.9
124 GMcD reference reference 19:03 1.11
125 Squallaby reference reference 19:04 1.47
126 chamelien tracked reference 19:10 0.75
127 rupe tracked reference 19:15 1.24
128 tapbarker tracked reference 19:32 0.79
129 Golvellius tracked reference 19:33 1.56
130 Craven reference reference 19:34 0.76
131 LuckOfLucien tracked reference 19:41 1.01
132 Pam tracked tracked 19:45 1.72
133 PHC tracked tracked 19:47 1.61
134 kath tracked reference 19:55 0.63
135 KET tracked reference 20:00 1.51
136 annelambirth reference reference 20:12 1.09
137 grape tracked reference 20:16 1.2
138 andrewlake tracked reference 20:31 1.1
139 Looch reference reference 20:37 0.79
140 pterodactyl reference reference 21:08 0.92
141 MichelinPoitiers new_solver reference 21:09 0.51
142 jimwetzler tracked reference 21:16 0.95
143 barracuda reference reference 21:17 0.9
144 Ed freshpot reference reference 21:22 0.95
145 Hamish.S new_solver tracked 21:23 0.44
146 poskir reference tracked 21:28 0.24
147 taberdar tracked tracked 21:31 1.76
148 mcjwright reference reference 21:37 1.19
149 sampson reference reference 21:51 0.72
150 X Solver 475 new_solver reference 21:54 1.28
151 Kevin Gregg reference reference 21:59 1.26
152 Zabadak reference reference 22:07 1.03
153 benedick reference reference 22:12 1.11
154 cath23 tracked reference 22:13 0.93
155 John Arnold reference reference 22:37 0.86
156 Solver52 reference tracked 22:52 0.42
157 brian.mcnamara new_solver reference 23:03 0.97
158 Anna Lancefield new_solver reference 23:05 1.17
159 jaraab tracked reference 23:06 1.11
160 puzzled1 tracked reference 23:07 0.52
161 gbestanley reference reference 23:11 0.51
162 Pootle new_solver reference 23:20 0.92
163 hugh.schofield tracked reference 23:45 1.28
164 flounderbout new_solver reference 23:47 0.72
165 colin2355 reference reference 24:01 1.13
166 george.garlick new_solver reference 24:16 1.01
167 Speedcop2 new_solver reference 24:21 1.56
168 Zander tracked reference 24:40 0.7
169 tonym tracked reference 25:10 0.87
170 frost tracked reference 25:18 1.45
171 zythum reference reference 25:25 1.0
172 Bert Dove tracked reference 25:56 0.88
173 timbo tracked reference 26:02 0.76
174 pc8 tracked reference 26:04 1.59
175 Dzhigit tracked reference 26:06 0.85
176 Mike Harper tracked tracked 26:32 2.2
177 rjcassidy reference reference 26:37 1.14
178 Yatton50 tracked reference 26:39 1.55
179 Chadwick reference reference 26:42 0.54
180 dec reference reference 26:45 0.84
181 Edprof reference reference 27:12 1.24
182 fudgiebates reference reference 27:12 1.47
183 GeorgeTz new_solver reference 27:15 0.84
184 goblin tracked reference 27:19 0.71
185 endafk tracked reference 27:21 1.18
186 diogenes reference reference 27:30 0.96
187 dont_panic reference reference 27:37 1.25
188 Jagsfan1971 tracked reference 27:48 1.34
189 chrissiemac tracked tracked 28:11 0.37
190 phmfantom reference reference 28:21 1.1
191 William M reference tracked 28:26 1.61
192 Manveru reference reference 28:30 0.74
193 cassandra reference tracked 28:47 0.19
194 jdavidliddell new_solver reference 28:58 0.87
195 Peter Sykes tracked reference 29:15 0.73
196 NAPASAI reference reference 29:18 1.05
197 Major S tracked reference 29:23 1.19
198 James P tracked reference 29:28 1.46
199 Paul H. Dear tracked reference 29:34 1.29
200 iagmathewson reference reference 29:37 0.49
201 Clovis tracked tracked 29:37 0.84
202 alwsn8 new_solver reference 29:53 1.12
203 bobbb tracked reference 31:30 1.31
204 JoeyC tracked reference 31:33 1.12
205 nekabue reference reference 31:44 1.16
206 HanS tracked tracked 31:48 0.17
207 Valance tracked reference 32:26 1.25
208 Agsurfer tracked tracked 32:38 0.4
209 Glyndwr reference reference 32:50 1.19
210 ulaca reference reference 33:10 1.29
211 Semillon tracked reference 33:19 1.26
212 JE59 tracked tracked 33:25 0.22
213 Mkgnao reference reference 33:33 1.36
214 Big Nug reference reference 33:38 0.9
215 FelicityTT reference reference 33:40 0.97
216 stainl665 new_solver reference 33:44 0.76
217 SwissWavey new_solver tracked 34:17 1.13
218 koro reference reference 34:37 0.96
219 Crippens1 reference reference 35:12 0.97
220 Jeeves reference reference 35:17 1.31
221 rosselliot reference reference 35:22 0.99
222 auntie noele tracked reference 35:25 1.51
223 Tudds reference reference 35:37 1.34
224 MrGrumpy tracked reference 35:58 0.8
225 eniamretrauq reference reference 36:13 1.01
226 SlumdogEngineer tracked reference 36:32 1.5
227 orieltonhouse tracked tracked 36:34 -0.86
228 maryht tracked tracked 36:34 0.3
229 JohnDun reference reference 36:40 1.11
230 joss henry tracked reference 37:10 1.08
231 1234 reference reference 37:29 0.8
232 Archie tracked reference 38:33 1.29
233 Nangle tracked reference 38:45 0.89
234 Pleasuredome tracked reference 38:54 0.71
235 Gussie new_solver reference 39:53 0.99
236 Paulypops tracked reference 40:24 1.04
237 empty reference tracked 40:40 0.4
238 Curarist tracked tracked 40:42 0.91
239 6098mr tracked tracked 41:05 0.01
240 Maggot tracked reference 41:58 1.31
241 oetjes tracked tracked 42:17 0.43
242 andrewmr tracked reference 43:27 1.11
243 Dandy tracked reference 43:31 0.66
244 dannyduncs new_solver tracked 43:54 -0.03
245 vissa tracked reference 43:57 0.61
246 Scottie tracked reference 44:04 1.43
247 SK42 tracked reference 44:08 0.61
248 Ronnie tracked reference 44:20 0.73
249 oldblueeyes tracked tracked 44:56 -0.17
250 symbenall2 tracked reference 44:59 1.19
251 MrChumley tracked tracked 45:16 0.75
252 MartinP tracked tracked 45:23 0.91
253 Msimmo new_solver tracked 46:03 0.86
254 RaghuS tracked tracked 47:23 0.65
255 starstruck tracked tracked 47:41 1.13
256 FrameShift tracked tracked 47:42 0.51
257 VictoriaW tracked tracked 47:44 0.02
258 paulmcl tracked tracked 49:26 0.44
259 Masha tracked tracked 49:35 0.65
260 Shushannah tracked tracked 50:42 0.95
261 kjbr tracked tracked 51:15 1.2
262 Gilzean tracked tracked 51:59 0.91
263 jackk.larg new_solver tracked 55:46 0.84
264 peebee1945 tracked tracked 56:51 1.07
265 kennethk new_solver tracked 57:35 0.63