Crossword SNITCH

A quantitative guide to the degree of difficulty of the Times Cryptic Crossword

Solver Types and a Neutrino Detector

The SNITCH has always relied on a set of Reference solvers to calculate the difficulty of the crossword. Updating this list over time has led me to look at different types of solver. In particular, it has been important to detect and remove "Neutrinos" from the data (see further below). Once this is done we can look at other types of solver more clearly.

What's a Neutrino?

Neutrino – A person who manually solves the puzzle on paper, and then goes to the Crossword Club and types in the answers as rapidly as possible, either in order to give a high position on the leaderboard or because they are entering a typing competition. (Times for the Times glossary definition)

Have you noticed that the top of the Times Cryptic Crossword Club leaderboard is occupied by solvers who have some remarkable completion times? How does someone solve the puzzle in around 2 minutes, when it probably takes that long to just write in the answers?

On the Times for the Times blog site, we've come to know these solvers as "Neutrinos", a term coined by Tony Sever some years back to describe those people who like to occupy the top spots on the leaderboard without actually being top-notch solvers. The theory is that these people do the crossword on paper, then transcribe the answers on the Crossword Club interface.

And the big question is: how can we detect these? Some of them are pretty obvious: if you look at the results at the table on the right, the top solver in recent months is sudokulover with an average solving time of 02:22 (min:sec). Given that Mark Goodliffe (10+ times winner of The Times Crossword Championship) has an average solving time of around 6 minutes, we're very sure that this is not a real solving time. But it's much harder to tell when you're talking about average times in the range of 5 to 8 minutes. Who are the Neutrinos and who are the real solvers? This page tries to answer that question.

Clusters of solvers

The chart below shows a way to separate Neutrinos from other solvers. It plots each solver's average solving time (in seconds) on the x-axis against an "alignment to SNITCH" factor on the y-axis. This alignment factor is 1 when, on average, a solver's personal NITCH rating for a crossword is proportional to the SNITCH rating for that crossword. It is 0 when there is no alignment. (For those interested, more detail on this is provided below).

The blue dots on this chart represent the Reference solvers for the SNITCH as of October 2019. (Note that you can hover over each point to see the name of the individual solver.) As you'd expect, these are clustered around an alignment factor of 1 (generally between 0.5 and 1.5) regardless of their average solving time.

The red dots represent solvers who have been around for some time but are not Reference solvers. A group of these is spread across the range of average solving times and also spread about the SNITCH alignment factor of 1. But there's a cluster of red dots in the lower left corner that has low solving times and is spread around an alignment factor of 0. These are the Neutrinos.

This data suggests that any solver with a solving time of less than, say, 8 minutes (480 seconds) and a SNITCH alignment factor of, say, less than 0.5 could be safely classified as a Neutrino.

The grey dots represent new solvers since the Reference solver list was updated early in 2019. We can now see that some of these appear to be Neutrinos (Blue Moon, for example), while others - even with low solving times - appear to be real solvers (e.g. Lihar). It would be very difficult to make this assessment without checking the alignment with the SNITCH, so it clears up any speculation that might arise for these.

Of course, this analysis is not 100% guaranteed, but it looks like a good start.

Remaining Questions

It's curious to see that a couple of solvers (Paul Gilbert and Mike Harper) are "super-aligned" to the SNITCH, i.e. they seems to find harder puzzles especially hard. From looking at the underlying data, my guess is that this is due to a relatively small sample size with a few results pulling the alignment higher.

It is a bit hard to know what's going on with the low-alignment, higher solving average results. There are quite a few with an alignment of less than 0.4, including a handful of Reference solvers. These are presumably not Neutrinos in the Club definition sense (unless they are just very slow typists). But if they are genuine solvers, we'd probably expect them to align to the SNITCH more closely. Maybe there is some underlying inconsistency (e.g. there are actually several solvers of unequal ability doing different crosswords or the solver sometimes has help and sometimes doesn't). These are speculations with no real way to test them.

Blog Solvers: adding solving times from the blog

On the Times for the Times blog site, solvers regularly report their solving times. Where they have allowed it, we now add them to the list of solvers and track their progress. This is a new category of solver, the Blog Solver, for those who've reported solving times regularly over the last six months.

Notes to Blog Solvers:

  • the most reliable format for reporting times is with format "mm:ss" or as "x minutes" in the title or at the start of the entry
  • only the first main comment for each blogger is used; responses to other comments or a second comment is ignored.
  • to report errors, include "n errors" just after the time
  • If you just have a time it is assumed you have no errors unless you mention DNF or pink squares or giving up, etc, in the text
  • Examples of good time notifications are "38:30, 0 errors" and "39 minutes, 0 errors" in the comment title or as the first part of your comment
  • This info is also available using the "Blog Solver" button on the right hand side of the detailed crossword solving times for each crossword

Updating solver status

This analysis allows me to reclassify solvers, as shown in the chart below. One of my main motivations is to be able to update the list of Reference solvers reliably. I'd like to be able to include new solvers (especially good ones) without accidentally including Neutrinos. And I'd like to track other non-Reference and non-Neutrino solvers, even if their data isn't used for the SNITCH.

So I can now classify solvers as follows:

Neutrino
Those with fast solving times and low alignment to the SNITCH (shown in orange)
Reference Solvers
Those aligned to the SNITCH and having an average solving time of less than 45 mins (shown in blue)
Tracked Solvers
Those that don't meet the conditions for Reference Solvers but I can be fairly sure are not Neutrinos (shown in red)
Blog Solvers
Those that report their solving times in the Times for the Times site (shown in purple)

New solvers will come into the list over time and can be reclassified on my next update.

I've shown the old and new status in the full table to the right so that you can track the changes for each user.

So what's the SNITCH alignment factor?

This is actually pretty straightforward if you know a little maths. I plot the solver's individual NITCH scores agains the corresponding SNITCH for the same puzzle and measure the slope of the line of best fit. A slope of 1 indicates that the results are proportional, a slope of 0 means that there's no correlation.

For example, verlaine has results as shown in the chart below. Easier puzzles (according to the SNITCH) are on the left of the x-axis, harder ones on the right. Verlaine's NITCH results vary, but are generally higher as we progress from left to right. The overall slope of the line of best fit is 0.95, showing good alignment with the SNITCH on average.

In contrast, sudokulover has results as shown in the chart below. There is no correlation with the hardness of the puzzle according to the SNITCH. In fact, the slope is negative, meaning that on average the harder puzzles are easier for sudokulover than the easier ones. This lack of correlation is exactly what you'd expect if the solver is just typing in known answers.

Nothing is simple and we have some solvers with a mix of results like the ones shown in the chart below. There is a correlation with the SNITCH on some solutions but not on others. On the rest, the solver looks just like a Neutrino, with short response times regardless of the SNITCH value. Presumably they type the answers in directly on these shorter puzzles. For this reason, where we detect this sort of solver, we will mark it manually as a Neutrino.

Solving times for club members

The table below shows crossword club solvers who have submitted more than 20 solutions in the last six months, with an average solving time of less than 60 minutes for an error-free crossword. They are ranked by that solving time. It also indicates their status, including whether they are Reference solvers for calculation of the daily SNITCH.

Rank Solver Status - Oct 2019 Current Status Average Time SNITCH Alignment
1 HarishTK tracked neutrino 01:54 -0.01
2 2 Down tracked neutrino 02:11 0.01
3 sudokulover tracked neutrino 02:22 0.01
4 Trevor Cooke tracked neutrino 02:41 0.12
5 ian.rann tracked neutrino 02:51 0.04
6 Pam tracked neutrino 02:53 -0.0
7 TimA tracked neutrino 02:56 -0.09
8 William Corder tracked neutrino 02:59 0.08
9 Margaut tracked neutrino 03:11 -0.05
10 Tink56 new_solver neutrino 03:14 -0.02
11 Mick the DD new_solver neutrino 03:15 0.1
12 shannaford new_solver neutrino 03:27 -0.05
13 Stevethepole tracked neutrino 03:31 0.04
14 RobG tracked neutrino 03:35 -0.15
15 Gwyrfai tracked neutrino 03:40 0.03
16 Nigel1953 tracked neutrino 03:41 0.24
17 BrianS tracked neutrino 03:49 0.03
18 GolfDocAndy tracked neutrino 03:50 0.04
19 adriano tracked neutrino 03:52 0.19
20 Dunsolvin new_solver neutrino 03:56 0.09
21 dilkrish tracked neutrino 04:04 0.1
22 rjr520 tracked neutrino 04:10 0.14
23 thebarnets new_solver neutrino 04:13 -0.3
24 Gunnergray tracked neutrino 04:18 0.08
25 seamus.butler new_solver neutrino 04:23 0.09
26 edwards.robert1 tracked neutrino 04:29 0.02
27 Mikemon tracked neutrino 04:42 0.04
28 Balbec tracked neutrino 04:46 -0.04
29 ray lewis tracked neutrino 05:02 -0.02
30 Henry Wyatt tracked neutrino 05:05 0.04
31 ernim tracked neutrino 05:12 0.05
32 chaswil tracked neutrino 05:33 0.07
33 Magoo reference reference 05:57 0.81
34 C M Gilles new_solver neutrino 06:02 0.49
35 Pontius tracked neutrino 06:23 0.11
36 Eamon Ryan reference reference 06:23 0.93
37 Jason reference reference 06:47 1.22
38 Diskus tracked reference 06:52 0.75
39 Old Fella new_solver neutrino 06:55 -0.21
40 Splottite new_solver neutrino 07:13 0.4
41 Nathan Panning new_solver reference 07:15 0.86
42 verlaine reference reference 07:23 0.99
43 escat49 tracked reference 07:31 0.59
44 tiljo tracked neutrino 07:42 -0.01
45 dick.sands new_solver neutrino 07:50 1.05
46 mistigris reference reference 08:00 1.27
47 Contrarian2 tracked tracked 08:06 0.96
48 aphis99 reference reference 08:35 1.07
49 Paul Gilbert tracked reference 08:52 1.32
50 hilarym tracked reference 09:05 1.22
51 Derek Harrison reference neutrino 09:07 0.1
52 asaldanha reference reference 09:24 1.29
53 boulemiche tracked reference 09:26 0.95
54 greggybell tracked neutrino 09:27 2.96
55 mauefw new_solver reference 09:35 0.92
56 Midas reference reference 09:41 0.91
57 richard reference reference 10:01 1.23
58 lili tracked reference 10:12 1.11
59 AlanS reference reference 10:16 1.3
60 chindit reference reference 10:48 1.04
61 chriswords reference reference 10:57 0.67
62 Peter Lewiston tracked reference 11:06 1.08
63 scrabbler new_solver reference 11:11 0.99
64 jimpen reference tracked 11:21 0.3
65 glheard reference reference 11:43 0.98
66 lickert reference reference 11:44 1.0
67 ChuckB tracked reference 11:50 0.86
68 philjordan1147 new_solver reference 11:58 1.26
69 tiggis tracked reference 12:01 1.18
70 wwalker50 new_solver reference 12:03 0.86
71 Kinghedger tracked reference 12:27 1.06
72 Topical Tim reference reference 12:29 1.23
73 trux reference reference 12:31 0.87
74 SimonAnthony reference reference 12:32 1.01
75 p.wild reference reference 12:40 0.64
76 neilr reference reference 12:41 1.0
77 jim42078 tracked reference 12:42 0.78
78 canismiser new_solver tracked 12:54 1.73
79 andy tracked reference 12:55 1.06
80 keriothe_ reference reference 12:56 1.19
81 John Walsh tracked reference 13:03 0.67
82 noidea reference reference 13:06 0.81
83 Tom Stubbs reference reference 13:11 1.02
84 miked1961 tracked reference 13:17 0.78
85 Sheapey reference reference 13:25 1.24
86 Stavros reference reference 13:28 1.02
87 Askival tracked reference 13:28 1.45
88 BDDF new_solver reference 13:42 1.02
89 robertL tracked reference 13:55 1.21
90 Spur58 reference tracked 13:58 0.45
91 Ben Staveley reference reference 14:02 1.19
92 aphasic tracked reference 14:03 1.02
93 HenryBW tracked reference 14:15 1.24
94 jamesah tracked reference 14:17 0.75
95 Silva reference reference 14:17 1.1
96 fshephe reference reference 14:18 0.71
97 Groucho reference reference 14:18 0.8
98 BoxBrownie tracked reference 14:21 0.98
99 Big Mick tracked reference 14:23 0.52
100 Roscoe tracked reference 14:31 1.36
101 Carol tracked reference 14:44 1.4
102 MikeOsborne reference reference 14:47 0.91
103 appa new_solver reference 14:48 1.35
104 Spanna reference reference 14:53 0.7
105 KenG tracked reference 14:53 1.17
106 Tibthorpe reference reference 15:03 1.31
107 njnclarke tracked reference 15:11 0.91
108 Blue501 new_solver neutrino 15:13 0.4
109 Malcj reference reference 15:17 1.11
110 Sue Sweeper reference reference 15:22 1.41
111 qieduan new_solver reference 15:23 1.18
112 tilbee new_solver reference 15:27 0.86
113 xyster reference reference 15:30 1.11
114 Aloysius tracked reference 15:51 1.13
115 Crucifer tracked reference 15:52 1.21
116 tunflog tracked tracked 15:55 0.44
117 JeffCollins new_solver reference 15:58 1.17
118 doublecross tracked reference 16:00 1.16
119 sotira reference reference 16:09 1.12
120 Penfold_61 new_solver reference 16:12 0.79
121 David Crooks reference reference 16:20 1.06
122 grape tracked reference 16:21 1.09
123 cyrillic reference reference 16:26 1.13
124 mikegrocott reference reference 16:37 0.69
125 d.middleton new_solver reference 16:42 1.31
126 phthisis reference reference 16:44 0.8
127 MK76 tracked reference 16:56 1.26
128 oshinmark tracked reference 17:06 0.87
129 tapbarker tracked reference 17:12 1.11
130 fishernick79 tracked reference 17:22 1.31
131 johninterred new_solver blog_solver 17:25 1.32
132 jimwetzler tracked reference 17:48 1.14
133 Pavel Tudor new_solver tracked 17:50 1.81
134 Ed freshpot reference reference 17:59 0.95
135 Golvellius tracked reference 18:01 1.36
136 JezWoolfrey reference reference 18:03 1.23
137 Dzhigit tracked reference 18:04 1.16
138 LouWeed reference reference 18:09 0.84
139 LuckOfLucien tracked reference 18:18 1.07
140 OliviaRhineb reference reference 18:19 1.06
141 Eddddddd tracked reference 18:26 1.04
142 Squallaby reference tracked 18:30 1.74
143 fuzz tracked reference 18:32 0.63
144 robinh reference reference 18:35 0.77
145 Mick in Iraq tracked reference 18:41 1.1
146 Brain new_solver tracked 18:43 0.15
147 Craven reference reference 18:46 0.9
148 gabrojak new_solver reference 18:50 1.01
149 RobR reference reference 18:55 1.14
150 GMcD reference reference 18:58 1.0
151 MichelinPoitiers new_solver tracked 19:10 0.38
152 Zinzan tracked tracked 19:11 1.71
153 LJCN new_solver reference 19:16 1.27
154 tombola tracked neutrino 19:16 1.89
155 annelambirth reference reference 19:28 1.01
156 rrricharrrd tracked tracked 19:38 1.98
157 Looch reference reference 19:42 0.89
158 pterodactyl reference reference 20:00 0.79
159 s_pugh tracked reference 20:13 1.05
160 flounderbout new_solver reference 20:14 1.1
161 jaraab tracked reference 20:23 0.87
162 Terrier new_solver reference 20:24 1.07
163 margaretl tracked reference 20:32 0.47
164 trishwain tracked reference 20:34 1.02
165 John Arnold reference reference 20:40 1.21
166 rupe tracked reference 20:41 1.58
167 Zabadak reference reference 20:42 1.04
168 Zander tracked reference 20:43 1.1
169 barracuda reference reference 20:53 0.95
170 benedick reference reference 20:59 0.82
171 cpn-byxtdjuh7 new_solver reference 21:01 1.11
172 Pootle new_solver reference 21:18 1.33
173 kath tracked reference 21:18 1.11
174 hugh.schofield tracked reference 21:21 1.19
175 mcjwright reference reference 21:24 1.07
176 Paul Bacon new_solver tracked 21:39 1.67
177 Hamish.S new_solver reference 21:42 1.48
178 peterh new_solver reference 21:49 1.03
179 colin2355 reference reference 21:51 1.22
180 tonym tracked reference 21:57 0.79
181 andrewlake tracked reference 22:00 1.15
182 Solver52 reference reference 22:17 0.49
183 sampson reference reference 22:44 0.89
184 Kevin Gregg reference reference 22:46 1.31
185 THE Engineer tracked tracked 23:00 1.71
186 george.garlick new_solver reference 23:04 0.85
187 poskir reference neutrino 23:04 0.18
188 gbestanley reference reference 23:06 0.49
189 puzzled1 tracked neutrino 23:13 0.2
190 Waterloo new_solver reference 23:13 1.15
191 Achernar tracked reference 23:36 1.46
192 timbo tracked tracked 23:39 1.68
193 jlancs228 new_solver reference 23:47 1.42
194 Bert Dove tracked reference 23:55 0.89
195 Big Nug reference reference 24:01 0.85
196 brian.mcnamara new_solver reference 24:03 1.12
197 KET tracked reference 24:06 1.19
198 fudgiebates reference reference 24:32 1.4
199 pc8 tracked reference 24:44 1.05
200 goblin tracked reference 24:48 0.74
201 endafk tracked reference 25:16 1.05
202 sue.cresswell new_solver reference 25:19 1.47
203 JeffHobbs tracked reference 25:26 1.39
204 dec reference reference 25:39 0.9
205 Jagsfan1971 tracked reference 25:40 1.17
206 Richard Bull tracked tracked 25:53 1.83
207 James P tracked reference 25:55 1.4
208 zythum reference reference 26:04 0.93
209 GeorgeTz new_solver reference 26:05 0.87
210 bobbb tracked reference 26:11 1.03
211 MartianAlien tracked tracked 26:11 0.38
212 Corymbia new_solver tracked 26:16 1.61
213 rjcassidy reference reference 26:25 1.13
214 ratterz tracked reference 26:37 1.29
215 joekobi new_solver blog_solver 26:38 1.2
216 crosswordian reference reference 26:40 0.97
217 klamery4 new_solver reference 26:41 0.64
218 Manveru reference reference 27:00 1.3
219 Semillon tracked reference 27:18 1.24
220 Edprof reference tracked 27:35 1.74
221 Chadwick reference reference 27:38 0.65
222 jdavidliddell new_solver reference 27:43 0.93
223 Clovis tracked tracked 27:43 0.74
224 peterjidavidson new_solver reference 27:44 0.97
225 annbrew tracked reference 28:04 0.97
226 eniamretrauq reference reference 28:21 1.12
227 William M reference reference 28:30 1.31
228 pph22 new_solver reference 28:38 0.72
229 alwsn8 new_solver reference 28:53 1.1
230 NAPASAI reference reference 28:59 1.47
231 Peter Sykes tracked reference 29:12 0.85
232 Tudds reference tracked 29:21 1.64
233 phmfantom reference reference 29:31 1.04
234 Mkgnao reference reference 29:32 0.89
235 alexh tracked reference 29:32 1.39
236 SlumdogEngineer tracked reference 29:43 1.26
237 nekabue reference reference 30:00 1.17
238 UisceTe new_solver reference 30:20 1.09
239 rosselliot reference reference 30:29 0.85
240 JE59 tracked reference 30:38 0.55
241 Stevemex tracked tracked 30:57 0.26
242 stainl665 new_solver reference 31:12 0.71
243 koro reference reference 31:37 0.89
244 Lloyd new_solver reference 31:41 1.38
245 Jeeves reference reference 31:42 1.38
246 JoeyC tracked reference 31:45 0.63
247 Agsurfer tracked tracked 31:50 0.14
248 Archie tracked reference 31:54 1.12
249 boltonwanderer new_solver blog_solver 31:56 1.6
250 JGP tracked reference 32:14 1.27
251 JohnDun reference reference 32:23 0.98
252 1234 reference reference 32:24 1.11
253 ulaca reference reference 32:41 1.46
254 SwissWavey new_solver tracked 32:43 1.15
255 Curarist tracked tracked 32:52 0.83
256 FelicityTT reference reference 32:55 1.11
257 MrGrumpy tracked reference 33:00 1.09
258 HanS tracked tracked 33:01 0.18
259 DMc128 new_solver reference 33:16 0.96
260 joss henry tracked reference 33:37 1.58
261 Crippens1 reference reference 34:19 1.13
262 Glyndwr reference reference 34:21 1.3
263 dosbarth new_solver reference 35:00 1.07
264 maryht tracked reference 35:04 0.55
265 john.brassey new_solver reference 35:07 0.65
266 Paulypops tracked reference 35:48 0.85
267 Jcal new_solver reference 36:42 1.11
268 Mike Harper tracked reference 36:53 1.04
269 Pleasuredome tracked reference 36:56 0.94
270 Nangle tracked reference 37:20 1.14
271 Gussie new_solver reference 37:24 0.83
272 Windsor53 new_solver reference 38:08 0.86
273 peterholgate new_solver reference 39:40 1.15
274 starstruck tracked tracked 40:02 1.15
275 Dandy tracked reference 40:12 1.39
276 Cookincats tracked reference 40:15 0.77
277 andrewmr tracked tracked 40:21 1.16
278 SK42 tracked reference 40:30 0.79
279 empty reference reference 41:11 0.48
280 symbenall2 tracked reference 41:20 1.11
281 wyn.dil new_solver reference 41:42 1.0
282 vissa tracked tracked 41:59 0.47
283 npweston new_solver reference 42:33 0.8
284 Conman666 tracked reference 43:58 0.77
285 Shushannah tracked reference 44:36 0.95
286 Msimmo new_solver tracked 45:14 0.88
287 WordPlodder new_solver tracked 45:15 0.96
288 MartinP tracked tracked 45:29 1.11
289 Ronnie tracked tracked 46:45 0.78
290 Dalai Farmer new_solver tracked 46:50 1.25
291 paulmcl tracked tracked 47:04 1.0
292 FrameShift tracked tracked 47:29 0.62
293 Masha tracked tracked 49:24 1.02
294 kjbr tracked tracked 49:42 0.78
295 jackk.larg new_solver tracked 52:59 1.04
296 marp new_solver tracked 53:01 0.81
297 kennethk new_solver tracked 54:31 0.83
298 VictoriaW tracked neutrino 55:08 -0.01
299 RaghuS tracked tracked 56:08 0.38