Crossword SNITCH

A quantitative guide to the degree of difficulty of the Times Cryptic Crossword

Solver Types and a Neutrino Detector

The SNITCH has always relied on a set of Reference solvers to calculate the difficulty of the crossword. Updating this list over time has led me to look at different types of solver. In particular, it has been important to detect and remove "Neutrinos" from the data (see further below). Once this is done we can look at other types of solver more clearly.

What's a Neutrino?

Neutrino – A person who manually solves the puzzle on paper, and then goes to the Crossword Club and types in the answers as rapidly as possible, either in order to give a high position on the leaderboard or because they are entering a typing competition. (Times for the Times glossary definition)

Have you noticed that the top of the Times Cryptic Crossword Club leaderboard is occupied by solvers who have some remarkable completion times? How does someone solve the puzzle in around 2 minutes, when it probably takes that long to just write in the answers?

On the Times for the Times blog site, we've come to know these solvers as "Neutrinos", a term coined by Tony Sever some years back to describe those people who like to occupy the top spots on the leaderboard without actually being top-notch solvers. The theory is that these people do the crossword on paper, then transcribe the answers on the Crossword Club interface.

And the big question is: how can we detect these? Some of them are pretty obvious: if you look at the results at the table on the right, the top solver in recent months is sudokulover with an average solving time of 02:16 (min:sec). Given that Mark Goodliffe (10+ times winner of The Times Crossword Championship) has an average solving time of around 6 minutes, we're very sure that this is not a real solving time. But it's much harder to tell when you're talking about average times in the range of 5 to 8 minutes. Who are the Neutrinos and who are the real solvers? This page tries to answer that question.

Clusters of solvers

The chart below shows a way to separate Neutrinos from other solvers. It plots each solver's average solving time (in seconds) on the x-axis against an "alignment to SNITCH" factor on the y-axis. This alignment factor is 1 when, on average, a solver's personal NITCH rating for a crossword is proportional to the SNITCH rating for that crossword. It is 0 when there is no alignment. (For those interested, more detail on this is provided below).

The blue dots on this chart represent the Reference solvers for the SNITCH as of October 2019. (Note that you can hover over each point to see the name of the individual solver.) As you'd expect, these are clustered around an alignment factor of 1 (generally between 0.5 and 1.5) regardless of their average solving time.

The red dots represent solvers who have been around for some time but are not Reference solvers. A group of these is spread across the range of average solving times and also spread about the SNITCH alignment factor of 1. But there's a cluster of red dots in the lower left corner that has low solving times and is spread around an alignment factor of 0. These are the Neutrinos.

This data suggests that any solver with a solving time of less than, say, 8 minutes (480 seconds) and a SNITCH alignment factor of, say, less than 0.5 could be safely classified as a Neutrino.

The grey dots represent new solvers since the Reference solver list was updated early in 2019. We can now see that some of these appear to be Neutrinos (Blue Moon, for example), while others - even with low solving times - appear to be real solvers (e.g. Lihar). It would be very difficult to make this assessment without checking the alignment with the SNITCH, so it clears up any speculation that might arise for these.

Of course, this analysis is not 100% guaranteed, but it looks like a good start.

Remaining Questions

It's curious to see that a couple of solvers (Paul Gilbert and Mike Harper) are "super-aligned" to the SNITCH, i.e. they seems to find harder puzzles especially hard. From looking at the underlying data, my guess is that this is due to a relatively small sample size with a few results pulling the alignment higher.

It is a bit hard to know what's going on with the low-alignment, higher solving average results. There are quite a few with an alignment of less than 0.4, including a handful of Reference solvers. These are presumably not Neutrinos in the Club definition sense (unless they are just very slow typists). But if they are genuine solvers, we'd probably expect them to align to the SNITCH more closely. Maybe there is some underlying inconsistency (e.g. there are actually several solvers of unequal ability doing different crosswords or the solver sometimes has help and sometimes doesn't). These are speculations with no real way to test them.

Blog Solvers: adding solving times from the blog

On the Times for the Times blog site, solvers regularly report their solving times. Where they have allowed it, we now add them to the list of solvers and track their progress. This is a new category of solver, the Blog Solver, for those who've reported solving times regularly over the last six months.

Notes to Blog Solvers:

  • the most reliable format for reporting times is with format "mm:ss" or as "x minutes" in the title or at the start of the entry
  • only the first main comment for each blogger is used; responses to other comments or a second comment is ignored.
  • to report errors, include "n errors" just after the time
  • If you just have a time it is assumed you have no errors unless you mention DNF or pink squares or giving up, etc, in the text
  • Examples of good time notifications are "38:30, 0 errors" and "39 minutes, 0 errors" in the comment title or as the first part of your comment
  • This info is also available using the "Blog Solver" button on the right hand side of the detailed crossword solving times for each crossword

Updating solver status

This analysis allows me to reclassify solvers, as shown in the chart below. One of my main motivations is to be able to update the list of Reference solvers reliably. I'd like to be able to include new solvers (especially good ones) without accidentally including Neutrinos. And I'd like to track other non-Reference and non-Neutrino solvers, even if their data isn't used for the SNITCH.

So I can now classify solvers as follows:

Neutrino
Those with fast solving times and low alignment to the SNITCH (shown in orange)
Reference Solvers
Those aligned to the SNITCH and having an average solving time of less than 45 mins (shown in blue)
Tracked Solvers
Those that don't meet the conditions for Reference Solvers but I can be fairly sure are not Neutrinos (shown in red)
Blog Solvers
Those that report their solving times in the Times for the Times site (shown in purple)

New solvers will come into the list over time and can be reclassified on my next update.

I've shown the old and new status in the full table to the right so that you can track the changes for each user.

So what's the SNITCH alignment factor?

This is actually pretty straightforward if you know a little maths. I plot the solver's individual NITCH scores agains the corresponding SNITCH for the same puzzle and measure the slope of the line of best fit. A slope of 1 indicates that the results are proportional, a slope of 0 means that there's no correlation.

For example, verlaine has results as shown in the chart below. Easier puzzles (according to the SNITCH) are on the left of the x-axis, harder ones on the right. Verlaine's NITCH results vary, but are generally higher as we progress from left to right. The overall slope of the line of best fit is 0.95, showing good alignment with the SNITCH on average.

In contrast, sudokulover has results as shown in the chart below. There is no correlation with the hardness of the puzzle according to the SNITCH. In fact, the slope is negative, meaning that on average the harder puzzles are easier for sudokulover than the easier ones. This lack of correlation is exactly what you'd expect if the solver is just typing in known answers.

Nothing is simple and we have some solvers with a mix of results like the ones shown in the chart below. There is a correlation with the SNITCH on some solutions but not on others. On the rest, the solver looks just like a Neutrino, with short response times regardless of the SNITCH value. Presumably they type the answers in directly on these shorter puzzles. For this reason, where we detect this sort of solver, we will mark it manually as a Neutrino.

Solving times for club members

The table below shows crossword club solvers who have submitted more than 20 solutions in the last six months, with an average solving time of less than 60 minutes for an error-free crossword. They are ranked by that solving time. It also indicates their status, including whether they are Reference solvers for calculation of the daily SNITCH.

Rank Solver Status - Oct 2019 Current Status Average Time SNITCH Alignment
1 HarishTK tracked neutrino 01:55 0.01
2 2 Down tracked neutrino 02:10 0.09
3 sudokulover tracked neutrino 02:16 -0.0
4 Trevor Cooke tracked neutrino 02:35 0.12
5 Margaut tracked neutrino 02:50 0.12
6 TimA tracked neutrino 02:55 0.19
7 William Corder tracked neutrino 02:57 0.12
8 ian.rann tracked neutrino 02:59 0.02
9 Stevethepole tracked neutrino 03:24 0.04
10 FunnyBunny tracked neutrino 03:32 -0.22
11 Nigel1953 tracked neutrino 03:34 0.19
12 Rsedge new_solver neutrino 03:34 0.16
13 GolfDocAndy tracked neutrino 03:38 -0.02
14 BrianS tracked neutrino 03:45 -0.01
15 Cryptic2015 tracked neutrino 03:47 0.02
16 robertsonjon new_solver neutrino 03:47 3.11
17 Gwyrfai tracked neutrino 03:49 -0.0
18 Gunnergray tracked neutrino 03:56 -0.05
19 Dunsolvin new_solver neutrino 04:00 0.1
20 leo tracked neutrino 04:14 -0.06
21 edwards.robert1 tracked neutrino 04:24 0.01
22 rjr520 tracked neutrino 04:37 0.3
23 ray lewis tracked neutrino 04:42 -0.01
24 Mikemon tracked neutrino 04:48 0.06
25 chaswil tracked neutrino 05:33 0.12
26 Henry Wyatt tracked neutrino 05:41 -0.22
27 greggybell tracked neutrino 06:00 -0.06
28 Magoo reference reference 06:02 0.91
29 Pontius tracked neutrino 06:28 0.12
30 Eamon Ryan reference reference 06:34 0.92
31 Splottite new_solver neutrino 06:45 0.14
32 tiljo tracked neutrino 07:06 0.04
33 Jason reference reference 07:07 1.29
34 escat49 tracked reference 07:19 0.64
35 Avid Crossworder new_solver neutrino 07:39 1.19
36 verlaine reference reference 07:42 1.02
37 mistigris reference reference 08:02 1.08
38 Contrarian2 tracked tracked 08:37 1.75
39 aphis99 reference reference 08:57 1.13
40 thebarnets new_solver neutrino 09:24 0.44
41 Midas reference reference 09:40 0.77
42 richard reference reference 09:49 1.17
43 asaldanha reference reference 09:50 1.08
44 hilarym tracked reference 09:54 0.91
45 Pam tracked tracked 09:55 2.04
46 boulemiche tracked reference 10:15 0.78
47 lili tracked reference 10:35 1.11
48 Paul Gilbert tracked tracked 10:36 1.69
49 chindit reference reference 10:44 0.99
50 Derek Harrison reference tracked 11:06 0.44
51 AlanS reference reference 11:08 1.1
52 jimpen reference tracked 11:15 0.32
53 philjordan1147 new_solver reference 11:20 1.51
54 chriswords reference reference 11:40 0.82
55 lickert reference reference 11:46 1.03
56 scrabbler new_solver reference 11:46 0.92
57 glheard reference reference 12:00 0.75
58 ChuckB tracked reference 12:02 0.86
59 tiggis tracked reference 12:20 1.24
60 colfal tracked tracked 12:28 0.31
61 robertL tracked reference 12:32 0.87
62 keriothe_ reference reference 12:44 0.97
63 Heyesey tracked reference 12:50 1.59
64 Kinghedger tracked reference 12:52 0.81
65 SimonAnthony reference reference 12:57 1.24
66 Tom Stubbs reference reference 13:00 0.96
67 trux reference reference 13:02 0.85
68 noidea reference reference 13:04 0.68
69 andy tracked reference 13:12 1.03
70 p.wild reference reference 13:19 0.55
71 neilr reference reference 13:27 1.19
72 jim42078 tracked reference 13:30 0.53
73 Topical Tim reference reference 13:32 1.32
74 Spur58 reference tracked 13:35 0.37
75 Peter Lewiston tracked reference 13:45 1.43
76 Sheapey reference reference 13:45 1.14
77 fshephe reference reference 13:48 0.65
78 John Walsh tracked reference 13:57 0.46
79 Silva reference reference 14:05 1.0
80 Stavros reference reference 14:17 0.96
81 BoxBrownie tracked reference 14:19 0.94
82 aphasic tracked reference 14:29 1.28
83 Groucho reference reference 14:31 0.93
84 Ben Staveley reference reference 14:32 0.84
85 Big Mick tracked reference 14:51 0.66
86 canismiser new_solver tracked 14:59 1.74
87 Malcj reference reference 15:38 1.0
88 Tibthorpe reference reference 15:41 1.46
89 mikegrocott reference reference 15:43 0.72
90 njnclarke tracked tracked 15:45 1.64
91 MikeOsborne reference reference 15:49 0.79
92 tunflog tracked tracked 15:54 0.3
93 KenG tracked reference 15:56 1.31
94 Spanna reference reference 16:02 0.66
95 Sue Sweeper reference reference 16:05 1.53
96 Eddddddd tracked reference 16:12 0.99
97 Crucifer tracked reference 16:25 1.03
98 cyrillic reference reference 16:26 0.99
99 phthisis reference reference 16:34 0.9
100 jonathanbaker174 tracked reference 16:37 1.2
101 HenryBW tracked reference 16:50 1.25
102 MK76 tracked reference 16:53 1.27
103 bigmack68 new_solver reference 16:58 1.32
104 David Crooks reference reference 17:14 1.06
105 sotira reference reference 17:16 0.99
106 margaretl tracked tracked 17:46 0.33
107 OliviaRhineb reference reference 18:17 1.0
108 LouWeed reference reference 18:18 0.84
109 LJCN new_solver reference 18:28 0.83
110 robinh reference reference 18:30 0.77
111 oshinmark tracked reference 18:38 0.9
112 Carol tracked reference 18:42 1.11
113 chamelien tracked reference 18:46 0.89
114 fuzz tracked tracked 18:51 0.42
115 Squallaby reference reference 18:52 1.42
116 RobR reference reference 18:57 1.2
117 Craven reference reference 18:58 0.87
118 tapbarker tracked reference 18:59 0.98
119 grape tracked reference 19:22 1.25
120 GMcD reference reference 19:34 1.03
121 annelambirth reference reference 19:40 1.16
122 jlancs228 new_solver reference 19:43 0.97
123 Ed freshpot reference reference 19:48 1.02
124 Golvellius tracked reference 19:51 1.58
125 Looch reference reference 19:52 0.83
126 Dixiegrand tracked reference 19:58 0.47
127 johninterred new_solver blog_solver 20:00 1.56
128 jimwetzler tracked reference 20:01 0.96
129 rupe tracked reference 20:08 1.46
130 LuckOfLucien tracked reference 20:15 1.01
131 pterodactyl reference reference 20:24 0.74
132 Hamish.S new_solver reference 20:24 0.86
133 MichelinPoitiers new_solver reference 20:38 0.58
134 kath tracked reference 21:09 0.72
135 rrricharrrd tracked reference 21:11 1.3
136 andrewlake tracked reference 21:15 0.92
137 Zander tracked reference 21:30 0.67
138 Dzhigit tracked reference 21:37 0.94
139 gbestanley reference reference 21:46 0.48
140 PHC tracked reference 21:47 1.39
141 Pootle new_solver reference 21:58 1.29
142 mcjwright reference reference 22:08 1.02
143 barracuda reference reference 22:09 0.83
144 benedick reference reference 22:14 1.0
145 cath23 tracked reference 22:14 1.08
146 flounderbout new_solver reference 22:17 1.01
147 EdP tracked tracked 22:20 0.11
148 Solver52 reference reference 22:39 0.57
149 howardwilde new_solver reference 22:42 1.31
150 John Arnold reference reference 22:45 0.95
151 sampson reference reference 22:47 0.78
152 george.garlick new_solver reference 22:56 0.98
153 Achernar tracked reference 23:00 1.16
154 Kevin Gregg reference reference 23:09 1.11
155 THE Engineer tracked tracked 23:11 2.28
156 peterh new_solver reference 23:11 1.32
157 jaraab tracked reference 23:14 0.96
158 poskir reference tracked 23:14 0.05
159 colin2355 reference reference 23:32 1.24
160 Zabadak reference reference 23:37 0.96
161 Anna Lancefield new_solver reference 23:41 1.23
162 puzzled1 tracked reference 23:43 0.47
163 X Solver 475 new_solver reference 23:47 1.12
164 brian.mcnamara new_solver reference 23:57 1.23
165 hugh.schofield tracked reference 24:47 1.45
166 Bert Dove tracked reference 25:09 1.24
167 KET tracked reference 25:18 1.02
168 goblin tracked reference 25:23 0.8
169 rjcassidy reference reference 25:26 1.12
170 dec reference reference 25:30 0.99
171 frost tracked reference 25:31 1.48
172 timbo tracked reference 25:59 1.01
173 MartianAlien tracked tracked 26:04 0.4
174 Speedcop2 new_solver reference 26:06 1.36
175 tonym tracked reference 26:07 0.88
176 GeorgeTz new_solver reference 26:36 0.76
177 endafk tracked reference 26:43 1.02
178 Waterloo new_solver reference 27:00 1.51
179 crosswordian reference reference 27:00 1.0
180 Chadwick reference reference 27:01 0.55
181 fudgiebates reference reference 27:05 1.14
182 Megistias tracked tracked 27:18 1.81
183 diogenes reference reference 27:32 1.05
184 zythum reference reference 27:45 1.01
185 pc8 tracked reference 27:49 1.5
186 Manveru reference reference 28:05 0.78
187 James P tracked reference 28:19 1.48
188 William M reference reference 28:24 1.38
189 Peter Sykes tracked reference 28:31 0.63
190 Jagsfan1971 tracked reference 28:40 1.43
191 jdavidliddell new_solver reference 28:46 0.99
192 phmfantom reference reference 28:50 1.0
193 dont_panic reference reference 29:25 1.16
194 Agsurfer tracked reference 29:45 0.48
195 Edprof reference reference 29:52 1.26
196 nekabue reference reference 30:05 1.1
197 Yatton50 tracked reference 30:07 1.13
198 rosselliot reference reference 30:46 1.0
199 alwsn8 new_solver reference 30:53 1.06
200 joekobi new_solver blog_solver 30:54 1.05
201 Mkgnao reference reference 31:09 0.97
202 docg new_solver reference 31:13 0.78
203 JoeyC tracked reference 31:26 1.39
204 NAPASAI reference reference 31:26 1.12
205 ratterz tracked tracked 31:29 1.64
206 HanS tracked tracked 31:34 0.16
207 ulaca reference reference 31:45 1.48
208 Clovis tracked tracked 31:46 0.78
209 Big Nug reference reference 31:48 1.0
210 boltonwanderer new_solver blog_solver 31:50 1.27
211 Paul H. Dear tracked reference 32:21 1.33
212 Semillon tracked reference 32:51 1.23
213 bobbb tracked reference 32:59 1.26
214 JE59 tracked reference 33:15 0.56
215 eniamretrauq reference reference 33:15 0.99
216 koro reference reference 33:26 1.08
217 Tudds reference reference 33:35 1.13
218 SlumdogEngineer tracked reference 33:37 1.03
219 FelicityTT reference reference 33:59 1.05
220 SwissWavey new_solver tracked 34:13 0.82
221 JohnDun reference reference 34:31 1.27
222 Jeeves reference reference 34:33 1.32
223 maryht tracked tracked 34:41 0.4
224 stainl665 new_solver tracked 34:41 0.43
225 klamery4 new_solver reference 34:43 0.76
226 Crippens1 reference reference 34:49 1.09
227 Glyndwr reference reference 35:04 1.17
228 joss henry tracked reference 35:35 1.14
229 MrGrumpy tracked reference 35:46 0.9
230 Archie tracked reference 36:23 1.02
231 1234 reference reference 36:34 0.97
232 6098mr tracked tracked 36:35 0.3
233 Curarist tracked tracked 38:28 0.79
234 Gussie new_solver reference 38:58 0.89
235 Mike Harper tracked reference 38:58 1.01
236 empty reference tracked 39:20 0.44
237 Paulypops tracked reference 39:27 0.99
238 oldblueeyes tracked tracked 40:30 0.19
239 Pleasuredome tracked reference 40:43 0.71
240 SK42 tracked reference 41:03 0.98
241 Nangle tracked reference 42:40 1.04
242 Dandy tracked reference 43:04 0.91
243 Scottie tracked tracked 43:13 1.89
244 Msimmo new_solver reference 43:56 0.89
245 symbenall2 tracked reference 44:05 1.36
246 Maggot tracked reference 44:35 1.36
247 Ronnie tracked reference 44:47 0.64
248 dannyduncs new_solver reference 44:52 0.48
249 starstruck tracked tracked 45:13 0.97
250 Conman666 tracked tracked 45:23 0.58
251 MartinP tracked tracked 45:25 1.15
252 MrChumley tracked tracked 46:00 0.81
253 vissa tracked tracked 46:47 -0.19
254 andrewmr tracked tracked 48:05 1.14
255 rick&max tracked tracked 48:13 0.73
256 Masha tracked tracked 48:17 1.1
257 Shushannah tracked tracked 48:44 0.8
258 RaghuS tracked tracked 48:45 0.41
259 paulmcl tracked tracked 48:53 0.98
260 FrameShift tracked tracked 49:49 0.59
261 VictoriaW tracked tracked 53:48 -0.02
262 kjbr tracked tracked 54:26 0.95
263 marp new_solver tracked 56:39 0.83
264 kennethk new_solver tracked 57:26 0.58
265 jackk.larg new_solver tracked 58:26 0.54