A Thirty-Year-Old Man Today vs. a Thirty-Year-Old Man in 1884. “Today: Is working toward paying off a two-bedroom condo by age seventy-five, if all goes well. 1884: Owns a drafty, four-hundred-square-foot cabin by the crick. Loves it.” https://www.newyorker.com/humor/shouts-murmurs/a-thirty-year-old-man-today-vs-a-thirty-year-old-man-in-1884

I finally turned off GitHub Copilot yesterday. I’ve been using it for about a year on the ‘free for open-source maintainers’ tier. I was skeptical but didn’t want to dismiss it without a fair trial.

It has cost me more time than it has saved. It lets me type faster, which has been useful when writing tests where I’m testing a variety of permutations of an API to check error handling for all of the conditions.

I can recall three places where it has introduced bugs that took me more time to to debug than the total time saving:

The first was something that initially impressed me. I pasted the prose description of how to communicate with an Ethernet MAC into a comment and then wrote some method prototypes. It autocompleted the bodies. All very plausible looking. Only it managed to flip a bit in the MDIO read and write register commands. MDIO is basically a multiplexing system. You have two device registers exposed, one sets the command (read or write a specific internal register) and the other is the value. It got the read and write the wrong way around, so when I thought I was writing a value, I was actually reading. When I thought I was reading, I was actually seeing the value in the last register I thought I had written. It took two of us over a day to debug this. The fix was simple, but the bug was in the middle of correct-looking code. If I’d manually transcribed the command from the data sheet, I would not have got this wrong because I’d have triple checked it.

Another case it had inverted the condition in an if statement inside an error-handling path. The error handling was a rare case and was asymmetric. Hitting the if case when you wanted the else case was okay but the converse was not. Lots of debugging. I learned from this to read the generated code more carefully, but that increased cognitive load and eliminated most of the benefit. Typing code is not the bottleneck and if I have to think about what I want and then read carefully to check it really is what I want, I am slower.

Most recently, I was writing a simple binary search and insertion-deletion operations for a sorted array. I assumed that this was something that had hundreds of examples in the training data and so would be fine. It had all sorts of corner-case bugs. I eventually gave up fixing them and rewrote the code from scratch.

Last week I did some work on a remote machine where I hadn’t set up Copilot and I felt much more productive. Autocomplete was either correct or not present, so I was spending more time thinking about what to write. I don’t entirely trust this kind of subjective judgement, but it was a data point. Around the same time I wrote some code without clangd set up and that really hurt. It turns out I really rely on AST-aware completion to explore APIs. I had to look up more things in the documentation. Copilot was never good for this because it would just bullshit APIs, so something showing up in autocomplete didn’t mean it was real. This would be improved by using a feedback system to require autocomplete outputs to type check, but then they would take much longer to create (probably at least a 10x increase in LLM compute time) and wouldn’t complete fragments, so I don’t see a good path to being able to do this without tight coupling to the LSP server and possibly not even then.

Yesterday I was writing bits of the CHERIoT Programmers’ Guide and it kept autocompleting text in a different writing style, some of which was obviously plagiarised (when I’m describing precisely how to implement a specific, and not very common, lock type with a futex and the autocomplete is a paragraph of text with a lot of detail, I’m confident you don’t have more than one or two examples of that in the training set). It was distracting and annoying. I wrote much faster after turning it off.

So, after giving it a fair try, I have concluded that it is both a net decrease in productivity and probably an increase in legal liability.

Discussions I am not interested in having:

• You are holding it wrong. Using Copilot with this magic config setting / prompt tweak makes it better. At its absolute best, it was a small productivity increase, if it needs more effort to use, that will be offset.
• This other LLM is much better. I don’t care. The costs of the bullshitting far outweighed the benefits when it worked, to be better it would have to not bullshit, and that’s not something LLMs can do.
• It’s great for boilerplate! No. APIs that require every user to write the same code are broken. Fix them, don’t fill the world with more code using them that will need fixing when the APIs change.
• Don’t use LLMs for autocomplete, use them for dialogues about the code. Tried that. It’s worse than a rubber duck, which at least knows to stay silent when it doesn’t know what it’s talking about.

The one place Copilot was vaguely useful was hinting at missing abstractions (if it can autocomplete big chunks then my APIs required too much boilerplate and needed better abstractions). The place I thought it might be useful was spotting inconsistent API names and parameter orders but it was actually very bad at this (presumably because of the way it tokenises identifiers?). With a load of examples with consistent names, it would suggest things that didn't match the convention. After using three APIs that all passed the same parameters in the same order, it would suggest flipping the order for the fourth.

#GitHubCopilot #CHERIoT

If you're struggling, seek support.

Mindline Trans 0300 330 5468 (Fridays 8pm-midnight)
TRUK Listens 0800 009 6640 (8am-midnight daily) https://truklistens.org/
Switchboard 0800 0119 100 (10am-10pm daily) https://switchboard.lgbt/
Samaritans 116123 (24hrs every day) https://www.samaritans.org/

Mastodon #research 🚀

I’m working on my master’s thesis at the University of Vienna, and I’d love your input! My research focuses on sustaining long-term user activity on Mastodon, contributing to a more active Fediverse. If you have a few minutes, please take my short survey:
👉 https://sosci.univie.ac.at/MastodonActivity/

Your thoughts will contribute to meaningful research. Feel free to boost this post so we can reach as many users as possible. Thank you so much! 🙌

#Mastodon #Survey #FediverseResearch

I'm not crying, you're crying.

Let’s Not Forget Charles Dickens’s Other Christmas Ghost Stories!

He loved writing these. There are very many.

By Olivia Rutigliano

https://crimereads.com/lets-not-forget-charles-dickenss-other-christmas-ghost-stories/

There are several editions of Christmas Carol in the PG collection:
https://www.gutenberg.org/ebooks/46
https://www.gutenberg.org/ebooks/19337
https://www.gutenberg.org/ebooks/24022
https://www.gutenberg.org/ebooks/30368

#books #literature

“Wow you’re good at computers”

“I just click everything”

“But how did you know which menu it was?”

“I just click everything”

“But then you were able to find the correct thing in the drop-down”

“I just click everything”

“How did you learn the new system so fast?”

“I just click everything”

“You must have some deep systems knowledge”

“I just click everything”

We need to normalize JOMO

To unwind after work, I shall move into a different room to sit in different chair and stare at a different glowing rectangle while a larger glowing rectangle plays in the background

So I just spent $100 on a new belt the other day, that doesn’t fit..
A huge waist.

#dadjokes

“In the role playing game known as The Real World, ‘Straight White Male’ is the lowest difficulty setting there is.” https://kottke.org/12/05/straight-white-male-the-game-of-lifes-lowest-difficulty-setting

Hello kind people of Mastodon! Mastodon is a big part of my life, so I'm going to give finding work here a go as well.

I'm looking for a junior penetration tester or cyber security analyst role. Should you have any leads, or simply be able to provide info, please reach out.

To give you a thumbnail background sketch: I'm a former Lecturer in Anthropology pivoting to Cyber Security. I recently passed the #OSCP and loved it. I have interests in writing and advising to make pentesting more accessible, IoT pentesting, and purple team work.

While I am looking for a first role, I'm open to freelance, contract, and part-time roles.

Boosts will be noted and the favor returned in due course!

Thanks everyone.

#fedihireme #fedihire #fedijobs #getfedhired #jobhunt #gethired #jobhunt

Genie: "For releasing me, I grant you a wish. "

"There are 3 rules: I can't kill anyone, I can't make anyone fall in love, and I can't bring people back from the dead."

Me: "I wish developers would stop making new #JavaScript frameworks."

Genie: "There are 4 rules..." #webDev

Roses are red
Bird bones are hollow
“Non sequitir” means
“It doesn’t follow”

My friend has developed an invisible aeroplane but I can’t see it taking off.

#jokeoftheday #dadjokes #hearditbefore

Cashier: Do you need bags?

Me: Do any of us NEED anything?
Cashier: Sir, I have a liberal arts degree too.
Me: Plastic, please.

American admiral and computer scientist, (designed COBOL) Grace Hopper was born #OTD in 1906.

She created the first compiler, the A-0 System, in 1952. She was also one of the first programmers on the Harvard Mark I computer. Hopper popularized the term "debugging" in computing after discovering an actual moth causing a malfunction in the Mark II computer.

https://en.wikipedia.org/wiki/Grace_Hopper

#computerscience #mathematics #womeninStem