Matt,
#2. Should telemetry be opt-in/out? Why? Discuss the tradeoffs
I am a bit disappointed that you even have to ask this. On the one hand its probably a cultural thing (the American default being opt-out, the European default being opt-in), but on the other hand, after the recent FaceBook fiasco I would have thought there would be more “sensitivity” to thinking about privacy before jumping in with a de facto opt-in (and the potential that that choice brings).
Clearly, I am going to say that it should be opt-in, and here are a few reasons why (with the last one being the more important):
-
On a practical level, if there are millions of Caddy instances in the wild, as they upgrade you are going to get hammered with a lot of data? Are you sure you’re reporting infrastructure can cope? Then again, that’s your problem and not mine
-
You need to be very explicit and detail exactly what data you are collecting, what you are going to do with it, how long you are going to keep it, and how people can delete it (you cover the first one, but not in any detail, cover the second, but not the last two). I want to see a sample of the telemetry data that you send (digging through the pull request for the caddy source and looking at +4000 lines to work out what’s in your json payload is not my idea of fun… and pity those who can’t read go and will have no idea of what they’re looking at).
-
The law of unintended consequences. With the telemetry data that you are collecting, could it be used for nefarious purposes, now or in the future? You state that “Telemetry does NOT collect personal information”, which may be true, but what it will collect, perhaps directly or indirectly, is the fingerprint of a server running caddy, and that even phoning home to drop off the telemetry data itself could have consequences (unintended or otherwise).
I would therefore strongly urge you to have a rethink… make it opt-in… and for each subset of metrics you are collecting, make them explicitly opt-in (i.e. no only do you have to opt-in at a top-level, you then have to explicitly opt-in for each sub-set of telemetry data to be sent).
Where is the source code for the telemetrics server? Is this closed or open? If you’re getting all this data wouldn’t it be nice to see how you’re crunching it?
- What charts/plots/numbers/tables should be shown on the page where you look up your instance?
I’d be curious to see the MITM metrics. Just how prevalent is this?
- What would you do with access to the telemetry data? What kind of research makes you excited/intrigued?
I’d see some interesting trend analysis possibilities.