Frequency analysis on hundreds of billions of reports at Report URI: Bloom Filters

By Scott Helme

Have we seen this report before? It sounds like a simple question to ask of a service that collects and processes hundreds of millions of reports per day, and in many ways it is, but in some ways it isn't. Here's how a probabilistic data structure

Weekly Update 266

By Troy Hunt

Presently sponsored by: VMware

Well this is a totally different office view! I'm properly getting into working more on the acoustics and aesthetics to make this the most productive environment possible which means this week things are in a bit of disarray due to ongoing works. Speaking of disarray, I've

Censored Vaccine Card

CVS's pharmacies are fine, but I much prefer their [censored]s.

Pwned - The Collected Blog Posts of Troy Hunt (Preview)

By Troy Hunt

Presently sponsored by: VMware

We chose this photo for the cover because this was when it all started. 18-year old Troy, having just discovered the web in early 1995 and chomping at the bit to do something with it. The full tale of what I first did (and how disastrous it ultimately became), is

Dark Arts

You think, 'okay, THIS is an ideal use case for hardlinks!' but then 6 months later you're doing some extremely cursed Google search like 'javascript ext4' and wondering where things went wrong.

A New Day at Simwood

By Grahame Davies

By Grahame Davies The idea that Simwood might ever have a CEO other than Simon had rarely crossed my mind until discussions about how the future might look, and what my role in that might be, evolved. Of course, most companies have strategic aims and contingency plans which could lead to such an eventuality at […]

Clinical Trials

We don't need to do a clinical trial of this change because the standard of care is to adopt new ideas without doing clinical trials.

New Keycloak maintainer: Takashi Norimatsu

By Stian Thorgersen

We are extremely pleased to welcome Takashi Norimatsu as an official maintainer of Keycloak.

Takashi has contributed to Keycloak since 2017, with a focus on security features of OAuth 2.0 and OpenID Connect, such as PKCE, strong signature algorithms, and Certificate Bound Access Tokens. More recently, he has been leading development related to Financial-grade API (FAPI) in the FAPI special interest group. In addition he has been helping other developers in the area of API authorization, including giving presentations at multiple conferences.

Takashi will continue leading development of OAuth 2.0 and OpenID Connect security related features, with an initial focus on features needed to provide higher level of API security for enterprise scenarios.

Takashi works for Hitachi, Ltd. in Japan, which sees the real value of Keycloak especially in the API management area, allowing him to invest a significant portion of his time to the Keycloak project.

The Keycloak team is very exited about having Takashi join us as a maintainer, and we are looking forward to working more closely with Takashi going forward.

Tasmota thermostat

By [email protected] (RevK)

The tasmota code has a proper thermostat mode, but it is not in the pre-built code, so I wanted to work out a way to do this with simple rules.

There are examples, yay... But I was struggling to make sense of these.

A simple example, say I want on at 25C and off at 29C...

The obvious rules are something like this :-

ON DS18B20-1#temperature>29 DO Power 0 ENDON

ON DS18B20-1#temperature<25 DO Power 1 ENDON

This works, but every second or so it is doing Power 1 or Power 0 constantly, and reporting that on MQTT, which is a tad annoying. It is worse if you have triggers on power setting to send publish commands to other things, etc.

The other examples I have seen seem to have a similar problem, I think. Some seem to have a timer that reduces the chatter but still, I just wanted something simple that only actually logged something and only actually changed power setting, when the temperature threshold is crossed.

What I came up with is not complicated, and seems to work:-

ON DS18B20-1#temperature>%var1% do backlog var1 999;var2 25;power 0 ENDON

ON DS18B20-1#temperature<%var2% do backlog var2 -999;var1 29;power 1 ENDON

The trick is the this sets the var1/var2 meaning that the temperature is no longer triggering an event all the time once it has changed.

Bingo, it works, and is a pretty simple pair of rules. Probably worth using mem1/mem2 as the 25/29 values here to be tidier, but still. Not complicated, and reacts immediately when the temperature passes the relevant threshold.

It also means the sensor can be a separate Shelly from the relay easily, only sending the publish to change the relay on change, which is also a big improvement.

P.S. For those that do not know, a Shelly 1 is a small ESP8266 based relay/switch that can switch mains power (16A) and typically about £8 (the blue round box). It has good firmware anyway, but Tasmota is an alternative firmware (open source) which can run on most ESP8266 and ESP32 based smart devices, so can run on a Shelly 1. DS18B20 is a common temperature sensor, and more than one can be connected in parallel even (the black wire and metal tube). Shelly do an adaptor (needed for DS18B20 as they would otherwise be mains voltage) to connect to temperature sensors (the black round thing on the Shelly 1).

Weekly Update 265

By Troy Hunt

Presently sponsored by: VMware

I had a bunch of false starts with this one. I don't know if it was just OBS or something else, but we got there after several failed attempts and me resorting to reading Gov Parson's nutty tweets until it all started working. "Nutty"

Saferways, not Quickways - safety and accessibility, not speed

By danny

Oxford's "Quickways" schemes may make cycling some trips faster for some people, but the biggest gains from them will be making cycling safer and more accessible. I will cycle some routes more slowly if these schemes are implemented! So "Quickways" is a misnomer — "Saferways" would be much better. My previous post about these schemes […]

Mitsubishi Lossnay+GUG aircon (not actually working)

By [email protected] (RevK)

We have a bit of a mystery here - the brand new, expensive, ducted air-con in the house does not work!

So what does not work exactly?

Not actually able to cool a room!

The biggest issue is it cannot cool a room - not even close. (It can't heat one much either)

This is a temperature plot for my study, 3m x 3m, you don't get a lot smaller than that in terms of a room to air-con. A=Portable unit on, B=Portable unit off, C=Start two hour cool only full fan test.

The test was with the other room vent closed so the unit was only cooling one room, in to a closed empty room, no heating on anywhere in the house, and outside temp of 19C. Not a challenge for any air-con really, and as you can see, the cheap portable unit managed with no problem (even with the door open most of the day). And yes, the fan speed control does work, and was on full.

I expected at point C the room to cool at least as fast as the cheap portable unit. What happened is it just about managed 0.8C in two hours! It was blowing cold air (measured at 9C) but not as lot...

The fix - well, the unit only has a 100mm vent in to the room. This seems small I must admit, and at full fan like this it is noisy. So they are changing to a 150mm vents today...

Did it work? In short, no. The bigger vents are quieter, which is nice, but the same tests showed maybe 1.5C drop and bottoming out just below 21C after more than an hour. Compare these big 3.5kW systems to a much cheaper 1.5kW portable unit which is able to drop my study (and hall way as door was open) from 23C to 18C in 15 minutes. With the door open the new air-con cannot cool my room at all!

This is now waiting on Mitsubishi to explain and fix. I won't say who the installers yet as they have been pretty good. Let's give them the chance to get Mitsubishi to fix it all.

But how is this even possible?

The issue for me here is that they sell this system, even with the smaller 2x 100mm outflow vents. So forgetting, for a moment, whether the suppliers got it wrong for the rooms we are trying to cool (two rooms each unit, one of which is much bigger than this 3m x 3m study), we have the issue that this whole unit running at full cooling and full fan cannot cool a 3m x 3m room.

So what the hell is the this unit for - what size room are they expecting it to be used with, ever?

The change to 150mm vents actually meant them cutting out the larger holes as the cowl as it is only supplied with the 100mm outlets.

This leads me to think something else is wrong, but we have no idea what. How could this ever work anywhere?

Cannot set temperature!

The controller has options - either "return air" temperature measured at the lossnay unit, or the room temperature at the controller. The controller has a sensor in it. Given that we have two rooms per unit, we are using the temperature at the controller (which is not in the room with the return in it). This is all as per the manual.

Only it does not work! Cooling the room down to 19C (using the portable unit, obviously), it showed 24C. Indeed, it even went up from 23C while we did this. Using freezer spray actually on its sensor does nothing, still 24C. No clue where it is measuring but it is not in the room - we even tried putting hot air in the room that has the return air flow but that did not make it go up at all.

The fix: Well, again, waiting on Mitsubishi.

Watch this space.

Management reshuffle

By Simwood

By Simon Woodhead Whenever I post a blog about management changes, the click through rate is astonishing which I could take the wrong way! Today’s post is what some of you may have been looking for on those past occasions! TLDR: I’m stepping down as CEO, Grahame Davies is replacing me. Over 25 years I’ve […]

Unsolved Math Problems

After decades of studying the curve and the procedure that generates it, the consensus explanation is "it's just like that."

Looking at the outlying data points

One of the fun things I used to do at a prior job was to dig through the raw data points which were collected from the *many* servers we had. I could just pick an interesting-sounding metric and then would just ask the system to show me the top 10 or bottom 10 out of a population of thousands or more.

Once in a while, something neat would jump out from the data. Most of the servers would have their values all clusters together, but then a handful of them would be "out there" to the point of being obvious at a glance. This is a story about one of those times.

To get my feet wet with the whole "investigations" thing that we'd do, I started looking into this one day and chose one of the many temp sensors which were being logged by the servers. I forget exactly which one it was, for there were many. The CPUs logged it, the chassis logged it, the power supply logged it, and so on. There were so many choices.

Anyway, out of any given group of systems, some of them would be clearly hotter than the others. I don't remember exactly how much it was any more, but want to say it might have been as much as 10 degrees C hotter in some cases, which is quite a bit, particularly for something that's supposed to be tightly controlled. The thermal profiles of those machines were all supposed to be the same, so having some of them be way out there didn't make much sense.

I went looking for a reason. What could possibly explain this? First off, it didn't line up with any obvious pattern in the host names. The hosts were named for whichever team owned them, followed by a sequence number that was bumped any time a new one was provisioned. There was no signal to be found in that.

A teammate showed me how to see other interesting stuff about the physical hardware, like the actual location it had inside the data center suite (the actual room). One dimension was "row" and the other was "rack". Think of them as an (x, y) coordinate pair within a given space. Here, too, nothing obvious jumped out. The weird heat came from all over. It wasn't just a hot spot in the suite, in other words.

There was more, though. You could also see a "rack position" in this data, and now it started getting interesting. It seems that all of the really hot machines had a rack position of something low, like 04 or 05, and never anything higher than that. In fact, when looking at a whole rack, it was always the lowest number of all of the positions that were being used. Now this was worth chasing!

I asked the people actually there in the locations how the numbering worked: was it top-down or bottom-up? Top-down would imply that 04 or 05 was the topmost server and then maybe it was cold air sinking and warm air rising? But that made no sense, since these things were supposed to be FORCING air from front to back and not leaving it to chance. There should have also been a gradient across the entire rack, and there was not one to be seen. It was just this one position that was much hotter, and the rest were all pretty "chill" (sorry).

It's good that I asked, since they told me something surprising: position 04 or 05 or whatever was the *bottom* of the rack. Yes, somehow, the lowest machine in the rack was the hottest! They found this interesting, too, and wound up digging into it. Given that I was far away in a distant office, this was the best way for us to proceed.

The Actual Engineers came back a while later with the answer: it seemed that the server design included this assumption that there would be another server underneath it. That way, the bottom of any given server's tray would be "sealed" by something else underneath it. Do you see the problem yet?

The problem was that the bottommost server would have nothing below it but just open space until it got to the floor. That space would not move air efficiently, and it obviously mattered.

The solution was great: they came up with something that looked like a server for airflow purposes, but which didn't actually do anything. It was basically two very big paperboard pizza boxes side by side. They would then slip that into the rack right below the last server, or any time they pulled one from somewhere higher in the stack. This would keep the airflow doing its thing happily for whatever was right above it.

This rolled out and my hot spots disappeared. Fantastic!

Blamer.el: A Git blame plugin for Emacs inspired by VS Code’s GitLens plugin


Ask HN: Senior Devs, what career have you moved on to avoid impeding doom?


Being spied on by the makers of Pegasus, then lured them into a trap


The Facebook Papers: dozens of stories based on whistleblower docs dropped


We Memo All the Things (2020)


Apple's M1 Pro, M1 Max SoCs Investigated


Running SSH-MitM on Android Devices


Anakin (YC S21) is Hiring web/app scraping engineers (Remote)


Nine Raspberry Pis power this entire office


Star Labs StarLite Mk IV 11" Linux Laptop with Coreboot


Project SPHINX – When the USSR tried to change the computer (2019)


Luxe: Cross platform, rapid development game engine


Kerla: Monolithic kernel in Rust, aiming for Linux ABI compatibility


WWII ships raised from watery graves by volcanic activity


Show HN: I built a fake VS Code to browse live cricket score in office


National Living Wage set to rise to £9.50 an hour

The 6.6% pay rise for workers aged 23 years old and over is expected to be announced in the Budget.

Intel hopes to burn newly open-sourced AI debug tech into chips

By Agam Shah

Chipzilla's future dreams for ControlFlag

Intel Labs has big plans for a software tool called ControlFlag that uses artificial intelligence to scan through code and pick out errors.…

David Bowie: Pop-up shop opens in London

From music to fashion to art, the Heddon Street pop-up celebrates 75 years of David Bowie.

Recycling plastics does not work, says Boris Johnson

The Recycling Association responds by saying Boris Johnson has "completely lost the plastic plot".

Climate change: Pledge of $100bn annual aid slips to 2023

A key pledge on climate funding has still not been met, and the money is not sure to be there before 2023.

Saudi crown prince suggested killing King Abdullah, ex-official says

Mohammed bin Salman discussed assassinating the late King Abdullah in 2014, Saad al-Jabri says.

Emiliano Sala: Agent would have let unqualified pilot fly player, court hears

Willie McKay had a "preoccupation" with getting a pilot for Cardiff-Nantes journey, court hears.

Facebook whistleblower Frances Haugen faces UK Parliament

Frances Haugen, who leaked thousands of documents, is appearing before MPs working on online safety.

SolarWinds attacker on the move: Russia's Nobelium crew has trebled attacks targeting MSPs, cloud resellers, says Microsoft

By Gareth Corfield

Phishing and password spraying on the up

Russia's Nobelium group – fingered as being a Russian state actor by both the United States and Britain – has massively ramped up phishing and password spraying attempts against managed service providers (MSPs) and cloud resellers, Microsoft's security arm has warned.…

Peak District photographer captures 'one in a million' rainbow shot

The picture is widely praised online after the photographer posted it on social media.

Tesla slams into reverse, pulls latest beta of Full Self-Driving software from participating car owners

By Richard Speed

FSD rolled back to 10.2 after 'issues' found

Tesla has yanked the latest beta, 10.3, of its Full Self-Driving (FSD) software from participating car owners after boss Elon Musk noted the company was "seeing some issues" with the code.…

'Unvaccinated tennis stars can play in Australia'

Unvaccinated players will be able to play in the Australian Open but face hard quarantine and regular testing, according to a letter distributed to WTA players.

T20 World Cup: Temba Bavuma on making and chasing history for South Africa

Temba Bavuma has already made history by becoming South Africa captain. Now he is chasing more at the T20 World Cup.

HIV Scotland fined £10,000 for BCC email blunder identifying names of virus-carriers' patient-advocates

By Paul Kunert

'Serious deficiencies in tech and organisational measures'

The United Kingdom's data watchdog is calling on organisations to review their "bulk email practices" after a BCC blunder by HIV Scotland incurred a £10,000 fine for breaking data protection regulations.…

Insulate Britain: Protesters block east London roads

In a statement, the group said: "We won't stand by while the government kills our kids."

Gen.G vs. Cloud9 / 2021 World Championship - Quarter-Final / Post-Match Discussion

By /u/adz0r


Official page | Leaguepedia | Liquipedia | Live Discussion | | New to LoL

Gen.G 3-0 Cloud9

- Gen.G move on to the Semifinals and will face EDward Gaming!

- Cloud9 have been eliminated from the tournament.

GEN | Leaguepedia | Liquipedia | Website | Twitter | Facebook | YouTube
C9 | Leaguepedia | Liquipedia | Website | Twitter | Facebook | YouTube | Subreddit

MATCH 1: GEN vs. C9

Winner: Gen.G in 41m
Game Breakdown | Runes

Bans 1 Bans 2 G K T D/B
GEN leblanc irelia twisted fate rakan leona 81.4k 16 11 H1 H3 B8 E9 B10
C9 yuumi renekton miss fortune azir sett 72.2k 11 6 I2 O4 B5 C6 C7
GEN 16-11-38 vs 11-16-23 C9
Rascal kennen 3 5-4-4 TOP 3-1-5 4 jarvan iv Fudge
Clid xin zhao 2 3-1-10 JNG 2-3-4 1 lee sin Blaber
Bdd aatrox 3 2-2-4 MID 5-4-2 2 yasuo Perkz
Ruler aphelios 1 5-3-7 BOT 1-3-7 1 ziggs Zven
Life lulu 2 1-1-13 SUP 0-5-5 3 nautilus Vulcan

MATCH 2: C9 vs. GEN

Winner: Gen.G in 25m
Game Breakdown | Runes

Bans 1 Bans 2 G K T D/B
C9 aphelios nami jarvan iv azir kennen 38.7k 4 0 H1
GEN leblanc twisted fate yuumi irelia jayce 54.1k 20 10 C2 H3 M4 I5 B6
C9 4-20-8 vs 20-4-44 GEN
Fudge malphite 3 0-4-2 TOP 7-0-4 3 graves Rascal
Blaber poppy 2 0-4-2 JNG 9-1-6 1 lee sin Clid
Perkz sylas 3 4-6-0 MID 1-1-15 4 syndra Bdd
Zven miss fortune 1 0-2-1 BOT 3-1-7 2 jhin Ruler
Vulcan leona 2 0-4-3 SUP 0-1-14 1 rakan Life

MATCH 3: C9 vs. GEN

Winner: Gen.G in 34m
Game Breakdown | Runes

Bans 1 Bans 2 G K T D/B
C9 aphelios nami irelia kennen azir 58.9k 10 4 H1 H3
GEN leblanc twisted fate yuumi olaf xin zhao 63.9k 17 8 O2 C4 M5 M6 B7
C9 10-17-18 vs 17-10-41 GEN
Fudge graves 2 3-5-2 TOP 1-2-6 3 renekton Rascal
Blaber jarvan iv 3 3-2-6 JNG 7-3-4 1 lee sin Clid
Perkz syndra 3 0-2-5 MID 1-3-13 4 zoe Bdd
Zven miss fortune 1 3-4-1 BOT 6-0-6 2 jhin Ruler
Vulcan leona 2 1-4-4 SUP 2-2-12 1 rakan Life

Patch 11.19

This thread was created by the Post-Match Team.

submitted by /u/adz0r to r/leagueoflegends
[link] [comments]

Halo Infinite - Campaign Overview Trailer

By /u/SmashBrosFanalic

Halo Infinite - Campaign Overview Trailer submitted by /u/SmashBrosFanalic to r/halo
[link] [comments]

I got called a prostitute for wearing these :/

By /u/xhollylathamx

I got called a prostitute for wearing these :/ submitted by /u/xhollylathamx to r/teenagers
[link] [comments]

The shapeshifter, Alyson Tabbitha

By /u/MoniMokshith

The shapeshifter, Alyson Tabbitha submitted by /u/MoniMokshith to r/Damnthatsinteresting
[link] [comments]

Conte keen on Manchester United job as players doubt Solskjær’s capability | Manchester United

By /u/calupict

Conte keen on Manchester United job as players doubt Solskjær’s capability | Manchester United submitted by /u/calupict to r/reddevils
[link] [comments]


By /u/Appointment-Funny

Bruh submitted by /u/Appointment-Funny to r/gaming
[link] [comments]

So my parents own a restaurant and this is their latest food pic of stuffed chicken breasts for their socials. Should I let them know….?

By /u/jetchflosher

So my parents own a restaurant and this is their latest food pic of stuffed chicken breasts for their socials. Should I let them know….? submitted by /u/jetchflosher to r/funny
[link] [comments]

the octopus just wants to play...

By /u/BusyBusyBizzy

the octopus just wants to play... submitted by /u/BusyBusyBizzy to r/MadeMeSmile
[link] [comments]

Auditory Illusions: Hearing Lyrics Where There Are None (OC: MonotoneTim)

By /u/SnooCompliments9613

Auditory Illusions: Hearing Lyrics Where There Are None (OC: MonotoneTim) submitted by /u/SnooCompliments9613 to r/blackmagicfuckery
[link] [comments]

on a video of someone pretending to fall over

By /u/hambugrer

on a video of someone pretending to fall over submitted by /u/hambugrer to r/rareinsults
[link] [comments]

25F security guard. I was left doing night shift alone after two of my co workers didn’t show up. This is how he responds after I ask him a simple question. What would you do in this situation?

By /u/FitManufacturer7098

25F security guard. I was left doing night shift alone after two of my co workers didn’t show up. This is how he responds after I ask him a simple question. What would you do in this situation? submitted by /u/FitManufacturer7098 to r/antiwork
[link] [comments]

Here are the rivers in Africa

By /u/OzCaaa1

Here are the rivers in Africa submitted by /u/OzCaaa1 to r/interestingasfuck
[link] [comments]

What event made you realise that you’re not the main character?

By /u/Top_Doubt1835

It’s easy to think the word revolves around you, but every now and then something happens to remind us that we’re really not that important.

My example (bit of a dull one) was going under general anaesthetic and when I came back round 4hrs later, I was convinced it had only been 10s. Made me appreciate how quickly things can stop and the world will carry on just fine without you.

submitted by /u/Top_Doubt1835 to r/AskUK
[link] [comments]

Having sex with a pornstar probably isn't as good as it sounds

By /u/Samjagoprime

submitted by /u/Samjagoprime to r/Showerthoughts
[link] [comments]

FFS how should I respond to this...

By /u/wimpynoob

FFS how should I respond to this... submitted by /u/wimpynoob to r/antiwork
[link] [comments]



Really interesting library for building Web Components: it's tiny (just 350 lines of code), works directly in browsers without any compile or build step and makes very creative use of modern JavaScript features such as async generators.

Via Alex Russell

New HTTP standards for caching on the modern web

New HTTP standards for caching on the modern web

Cache-Status is a new HTTP header (RFC from August 2021) designed to provide better debugging information about which caches were involved in serving a request - "Cache-Status: Nginx; hit, Cloudflare; fwd=stale; fwd-status=304; collapsed; ttl=300" for example indicates that Nginx served a cache hit, then Cloudflare had a stale cached version so it revalidated from Nginx, got a 304 not modified, collapsed multiple requests (dogpile prevention) and plans to serve the new cached value for the next five minutes. Also described is $Target-Cache-Control: which allows different CDNs to respond to different headers and is already supported by Cloudflare and Akamai (Cloudflare-CDN-Cache-Control: and Akamai-Cache-Control:).

Via Hacker News

Why you shouldn't invoke directly

Why you shouldn't invoke directly

Paul Ganssle explains why you shouldn't use "python command" any more. I've mostly switched to pip and pytest and twine but I was still using "python sdist" - apparently the new replacement recipe for that is "python -m build".

Via @pganssle

Datasette 0.59: The annotated release notes

Datasette 0.59 is out, with a miscellaneous grab-bag of improvements. Here are the annotated release notes.

Column metadata

Columns can now have associated metadata descriptions in metadata.json, see Column descriptions. (#942)

I've been wanting this for ages. Tables consist of columns, and column names very rarely give you enough information to truly understand the associated data. You can now drop extra column definitions into your metadata.json like so:

  "databases": {
    "la-times": {
      "tables": {
        "cdph-county-cases-deaths": {
          "columns": {
            "county": "The name of the county where the agency is based.",
            "fips": "The FIPS code given to the county by the federal government. Can be used to merge with other data sources.",
            "date": "The date when the data were retrieved in ISO 8601 format.",
            "confirmed_cases": "The cumulative number of coronavirus cases that were confirmed as of that time. This is sometimes called the episode date by other sources.",
            "reported_cases": "The cumulative number of coronavirus cases that were reported as of that time. This reflects when cases were first announced by the state.",
            "probable_cases": "The cumulative number of probable coronavirus cases that were confirmed as of that time. This reflects the results of antigen tests, a rapid testing technique different from the standard test.",
            "reported_and_probable_cases": "The cumulative number of reported and probable coronavirus cases as of that time.",
            "reported_deaths": "The cumulative number of deaths reported at that time."

The LA Times publish a meticulous data dictionary for the 46 CSV and GeoJSON files that they maintain tracking the pandemic in their california-coronavirus-data GitHub repository.

To demonstrate the new column metadata feature, I wrote a script that converts their Markdown data dictionary into Datasette's metadata format and publishes it along with imported CSV data from their repository.

You can explore the result at - here's their cdcr-prison-totals table tracking Covid cases in prisons operated by the California Department of Corrections and Rehabilitation.

register_commands() plugin hook

New register_commands() plugin hook allows plugins to register additional Datasette CLI commands, e.g. datasette mycommand file.db. (#1449)

I originally built this because I thought I would need it for Datasette Desktop, then found it wasn't necessary for that project after all.

I held off on implementing this for quite a while on the basis that plugins which needed their own CLI interface could implement one entirely separately as a Click-based CLI app - dogsheep-beta implements this pattern, offering both a dogsheep-beta index command and registering itself as a Datasette plugin.

The problem with plugins implementing their own separate CLI commands is that users then need to understand where they have been installed. Datasette is designed to work well with virtual environments, but if a plugin is installed into a virtual environment I can't guarantee that any CLI tools it includes will execute when the user types their name.

Now that plugins can register datasette subcommand subcommands this problem has a solution: provided users can run datasette they'll also be able to run CLI commands provided by plugins, without needing to understand how to modify their path.

I expect this to be particularly useful for Datasette installed through Homebrew, which invisibly sets up its own virtual environment into which plugins can be installed using datasette install plugin-name.

Count unique facet values with ?_facet_size=max

Adding ?_facet_size=max to a table page now shows the number of unique values in each facet. (#1423)

When I'm using facets to explore data, I'm often interested in how many values are available in the facet - particularly if I'm faceting by a column such as country or state.

I added a ... link to show the maximim number of facets in Datasette 0.57. Clicking that link adds ?_facet_size=max to the URL, which now also adds a numeric count of the number of distinct facet values.

Here's an example using that Californian prison data.

Facet by name shows 35 values

Upgrading httpx

Upgraded dependency httpx 0.20 - the undocumented allow_redirects= parameter to datasette.client is now follow_redirects=, and defaults to False where it previously defaulted to True. (#1488)

A while ago Tom Christie requested feedback on how httpx should handle redirects.

The requests library that inspired it automatically follows 301 and 302 redirects unless you explicitly tell it not to.

I've been caught out by this many times in the past - it's not default behaviour that I want from my HTTP client library - so I chimed in as favouring a change in behaviour. I also suggested that follow_redirects=True would be a better term for it than allow_redirects=True.

Tom made that change for httpx 1.0, and then back-ported it for version 0.20 - after all, pre-1.0 you're allowed to make breaking changes like this.

... and Datasette broke, hard! Datasette embeds httpx pretty deeply inside itself, and the breaking change caused all kinds of errors and test failures.

This was the final push I needed to get 0.59 released.

(I just found out this also broke the Homebrew tests for Datasette, as those relied on datasette --get '/:memory:.json?sql=select+3*5' automatically following the redirect to /_memory.json?sql=select+3*5 instead.)

Code that figures out which named parameters a SQL query takes in order to display form fields for them is no longer confused by strings that contain colon characters. (#1421)

One of my favourite obscure features of Datasette is the way it can take a SQL query such as the following:

select * from [cdcr-prison-totals]
where "zipcode" = :zip

And extract out that :zip parameter and turn it into an HTML form field, as seen here:

A form showing the SQL query with a separate field for entering the zip

This used to use a regular expression, which meant that it could be confused by additional colons - the following SQL query for example:

select * from content
where created_time = '07:00'
and author = :author

I thought that solving this properly would require embedding a full SQLite-compatible SQL parser in Datasette.

Then I realized SQLite's explain output included exactly the data I needed, for example:

addr opcode p1 p2 p3 p4 p5 comment
0 Init 0 10 0 0
1 OpenRead 0 42 0 2 0
2 Rewind 0 9 0 0
3 Column 0 1 1 0
4 Ne 2 8 1 BINARY-8 82
5 Rowid 0 3 0 0
6 Column 0 1 4 0
7 ResultRow 3 2 0 0
8 Next 0 3 0 1
9 Halt 0 0 0 0
10 Transaction 0 0 35 0 1
11 Variable 1 2 0 :name 0
12 Goto 0 1 0 0

The trick then is to run an explain, then find any rows with an opcode of Variable and read the p4 register to find out the name of those variables.

This is risky, since SQLite makes no promises about the stability of the explain output - but it's such a useful trick that I'm now contemplating building an automated test suite around it such that if a future SQLite release breaks things I will at least know about it promptly.

Everything else

The --cors option now causes Datasette to return the Access-Control-Allow-Headers: Authorization header, in addition to Access-Control-Allow-Origin: *. (#1467)

This was a feature request from users of the datasette-auth-tokens plugin.

Renamed --help-config option to --help-settings. (#1431)

Part of my continuing goal to free up the term "config" to mean plugin configuration (which is currently mixed up with Datasette's metadata concept) rather than meaning the options that can be passed to the Datasette CLI tool (now called settings).

datasette.databases property is now a documented API. (#1443)

I've got into the habit of documenting any Datasette internals that I use in a plugin. In this case I needed it for datasette-block-robots.

The base.html template now wraps everything other than the <footer> in a <div class="not-footer"> element, to help with advanced CSS customization. (#1446)

I made this change so that Datasette Desktop could more easily implement a sticky footer that stuck to the bottom of the application window no matter how short or tall it was.

The render_cell() plugin hook can now return an awaitable function. This means the hook can execute SQL queries. (#1425)

Another example of the await me maybe pattern in action.

register_routes(datasette) plugin hook now accepts an optional datasette argument. (#1404)

This means plugins can conditionally register routes based on plugin configuration.

New hide_sql canned query option for defaulting to hiding the SQL query used by a canned query, see Additional canned query options. (#1422)

The goal of canned queries is to provide an interface for people who don't know SQL to execute queries written by other people, potentially providing their own inputs using the :parameter mechanism described above.

If it's being used for that, there's not much to be gained from making them scroll past the SQL query first!

Adding "hide_sql": true to the canned query configuration now defaults to hiding the query for them - though they can still click "Show SQL" to see it.

New --cpu option for datasette publish cloudrun. (#1420)

Google Cloud Run recently added the ability to specify 1, 2 or 4 vCPUs when running a deploy.

If Rich is installed in the same virtual environment as Datasette, it will be used to provide enhanced display of error tracebacks on the console. (#1416)

Rich is Will McGugan's phenomenal Python library for building beautiful console interfaces. One of the many tricks up its sleeve is improved display of exceptions, including more detailed tracebacks and local variables. Datasette now takes advantage of this if Rich is installed in the same virtual environment.

datasette.utils parse_metadata(content) function, used by the new datasette-remote-metadata plugin, is now a documented API. (#1405)

Another API that became documented after I used it in a plugin.

The datasette-remote-metadata plugin is pretty neat.

I sometimes find myself working on projects where I'm deploying a large database file - 1 or 2 GB - to Google Cloud Run. Each deploy can take several minutes.

If I want to tweak a canned query or a few lines of text in the metadata for that deployment, it can be frustrating to have to push an entirely new deploy just to make those changes.

The remote metadata plugin allows me to host the metadata at a separate URL, which I can then update without needing a full deploy of the underlying database files.

The last two were simple bug fixes:

  • Fixed bug where ?_next=x&_sort=rowid could throw an error. (#1470)
  • Column cog menu no longer shows the option to facet by a column that is already selected by the default facets in metadata. (#1469)

Releases this week

TIL this week

Where does all the effort go? Looking at Python core developer activity

Where does all the effort go? Looking at Python core developer activity

Łukasz Langa used Datasette to explore 28,780 pull requests made to the CPython GitHub repository, using some custom Python scripts (and sqlite-utils) to load in the data.

Via @llanga

Tests aren’t enough: Case study after adding type hints to urllib3

Tests aren’t enough: Case study after adding type hints to urllib3

Very thorough write-up by Seth Michael Larson describing what it took for the urllib3 Python library to fully embrace mypy and optional typing and what they learned along the way.

Web Browser Engineering

Web Browser Engineering

In progress free online book by Pavel Panchekha and Chris Harrelson that demonstrates how a web browser works by writing one from scratch using Python, tkinter and the DukPy wrapper around the Duktape JavaScript interpreter.

Via @humphd

How to win at CORS

How to win at CORS

Jake Archibald's definitive guide to CORS, including a handy CORS playground interactive tool. Also includes a useful history explaining why we need CORS in the first place.

Via @jaffathecake

Finding and reporting an asyncio bug in Python 3.10

I found a bug in Python 3.10 today! Some notes on how I found it and my process for handling it once I figured out what was going on.

Testing Datasette against Python 3.10

I finally got around to attempting to upgrade Datasette to the just-released Python 3.10 this morning. I started by adding "3.10" to the matrix of Python versions that Datasette is tested against by GitHub Actions:

    python-version: ["3.6", "3.7", "3.8", "3.9", "3.10"]

That line previously looked like this:

python-version: [3.6, 3.7, 3.8, 3.9]

I switched to using quoted strings here based on a tip from Jeff Triplett, who pointed out that 3.10 is actually a floating point number in YAML syntax which will be treated as referring to Python 3.1 instead!

This one-line change in a pull request was all it took to run the tests against the new version of Python... and they failed, spectacularly.

Annoyingly the root error wasn't displayed directly in the tests, because it was triggered and then caught by Datasette's own error handling and turned into a 500 HTTP error. So, I needed to run Python 3.10 locally and it the debugger.

Running Python 3.10 locally

I occasionally use pyenv to manage multiple Python versions on my machine. I have this installed through Homebrew.

I had to run brew upgrade pyenv first, because my installed version didn't know about Python 3.10. Then I ran this:

pyenv install 3.10.0

Which gave me a Python 3.10 installation on my Mac in ~/.pyenv/versions/3.10.0/bin/python.

Finally, I created a new virtual environment using this version of Python.

~/.pyenv/versions/3.10.0/bin/python \
  -m venv /tmp/py310
# Now activate that virtual environment:
source /tmp/py310/bin/activate

I put this in my /tmp directory so I don't have to remember to clean it up later.

Having done this, I installed my Datasette test dependencies into the new environment and used pytest to run my tests:

% cd ~/Dropbox/Development/datasette
% pip install -e '.[test]'

Running pytest -x --pdb stopped at the first failing test and dropped me into a debugger where I could finally access the full traceback, which looked something like this:

  File "datasette/datasette/views/", line 122, in dispatch_request
    await self.ds.refresh_schemas()
  File "datasette/datasette/", line 344, in refresh_schemas
    await self._refresh_schemas()
  File "datasette/datasette/", line 349, in _refresh_schemas
    await init_internal_db(internal_db)
  File "datasette/datasette/utils/", line 5, in init_internal_db
    await db.execute_write(
  File "datasette/datasette/", line 102, in execute_write
    return await self.execute_write_fn(_inner, block=block)
  File "datasette/datasette/", line 113, in execute_write_fn
    reply_queue = janus.Queue()
  File "janus/", line 39, in __init__
    self._async_not_empty = asyncio.Condition(self._async_mutex)
  File "lib/python3.10/asyncio/", line 234, in __init__
    raise ValueError("loop argument must agree with lock")
ValueError: loop argument must agree with lock

So, something is going long with asyncio.Condition(), which is being called by janus.Queue().

Some background on Janus

The Janus library referenced here describes itself as a "Thread-safe asyncio-aware queue for Python".

I use it in Datasette for the write queue - a mechanism where writes can be safely made to a SQLite database by first queueing them up in memory.

Janus provides a mechanism that lets asyncio event tasks send and receive messages to Python threads, and vice-versa, via a queue class that is modelled on Python's own standard library queues. It's really neat.

Tracking the investigation in issue comments

Any time I'm investigating a bug I make sure that there's an associated GitHub issue or pull request, to give me somewhere to build up detailed notes of my investigation.

I used my PR #1481 for this. Now that I had an error message - "loop argument must agree with lock" - I did some initial Google searches and found to my surprise that it had very few existing mentions - my first clue that this might be related to the new Python 3.10 release itself.

I tracked down the relevant source code in both Python and the Janus library based on the traceback and linked to them from the issue comments. I pasted in copies of the relevant code too, since GitHub only magically embeds code from the same repository, not code from other repositories. You can see those comments here.

I always like to include copies of the code most likely to be the cause of a bug in an issue comment, to save myself from having to dig around for it again later on and to act as a historical record once the bug has been fixed.

I also started an issue in the Janus repo, with the title Error with Python 3.10: "ValueError: loop argument must agree with lock". I linked this to my own issue and posted relevant research in both places.

The two lines of code in Janus that caused the bug where these ones (note my added comment):

self._async_mutex = asyncio.Lock()
# "loop argument must agree with lock" exception is raised here:
self._async_not_empty = asyncio.Condition(self._async_mutex)

This is where I began to suspect a Python bug. The above code can be simplified to this:


This is a documented way of using conditions in Python: you can instantiate conditions with an optional lock, which allows that lock to be shared by more than one condition.

So I tried that in the Python 3.10 interpreter... and it didn't throw the error:

% ~/.pyenv/versions/3.10.0/bin/python
Python 3.10.0 (default, Oct  7 2021, 13:45:58) [Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> print(asyncio.Condition(asyncio.Lock()))
<asyncio.locks.Condition object at 0x1062521d0 [unlocked]>

From browsing the source code I could see that there was something going on here involving the event loop - so on a hunch I tried running that code in an event loop instead, like so:

>>> async def example():
...     print(asyncio.Condition(asyncio.Lock()))
Traceback (most recent call last):
  File "<stdin>", line 2, in example
  File "/Users/simon/.pyenv/versions/3.10.0/lib/python3.10/asyncio/", line 234, in __init__
    raise ValueError("loop argument must agree with lock")
ValueError: loop argument must agree with lock

There's the exception!

So it looked like this might be a genuine Python 3.10 bug. After all, I knew that this code worked in prior versions of Python (since Janus and Datasette worked fine there). In fact I could confirm it like so:

~ % python3.9
Python 3.9.7 (default, Sep  3 2021, 12:45:31) 
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> async def example():
...     print(asyncio.Condition(asyncio.Lock()))
<asyncio.locks.Condition object at 0x1049aab50 [unlocked]>

Reporting the bug to the Python issue tracker

Python bugs are tracked on - thankfully they support sign in with GitHub (and Google and even OpenID) so I signed in with my GitHub account to report the bug.

I filed Issue 45416 entitled:

"loop argument must agree with lock" instantiating asyncio.Condition

And included a brief description with my minimal Python script that demonstrated the bug.

Then I added a comment pointing back to one of the related GitHub issues where I had been figuring this out, and another comment that linked to the most relevant test in Python's own test suite, which I found using GitHub search for the term Condition.

Any time I file an issue against a project - my own or someone else's - I always like to do a bit of extra research and link to both the likely code at fault and the likely test.

As a maintainer, this saves me a bunch of time - if the issue links straight to the code and tests I don't have to spend any time digging around looking for them.

My hope is that this saves time for the people I am reporting the bug to as well, which increases the chance that they'll dig into it and help me find a fix!

Then I tweeted about it. I have a bunch of followers with relevant experience, so I thought it was time to get a few more eyes on the problem.

I included a screenshot of my steps-to-reproduce script to save anyone who might be casually interested from having to click through to the issue itself.

Łukasz Langa to the rescue

Łukasz is the "CPython Developer in Residence", which means he is paid by the Python Software Foundation to work on Python full-time.

Less than an hour after I filed the issue he posted a comment that both confirmed the bug and provided two short-term workarounds for it!

I really liked his first workaround, which looks like this:

>>> l = asyncio.Lock()
>>> getattr(l, '_get_loop', lambda: None)()
<_UnixSelectorEventLoop running=True closed=False debug=False>

It works by calling the lock._get_loop() method, but only if it exists - so even though that's an undocumented internal method this should be safe even when run against future Python versions that might remove the method.

You can see that method here - it safely populates the self._loop property, which helps work around the bug.

Submitting a PR to Janus

The best place to apply this workaround is to Janus - so I submitted a PR there which adds the workaround and updates their CI configuration to test against Python 3.10: Janus PR #359. The GitHub Actions tests for that PR are now passing against Python 3.10.

Their README links to a gitter chat so I dropped a link to my PR in there too. Hopefully that can be merged soon!

What I should have done

I'm feeling pretty good about this situation - the bug is set to be fixed, we have a great workaround thanks to Łukasz and I'm optimistic that the upstream library from Datasette will land the fix soon.

If only I'd done this a few weeks earlier!

The Python 3.10 release schedule shipped the first alpha a year ago, in October 2020. It totalled 7 alphas, 4 betas and 2 release candidates before the final 3.10 release on Monday 4th October 2021.

If I'd taken the time to get Datasette's test suite running against Python 3.10 for any of those pre-releases, I could have helped spot this bug before it landed in the final release!

So lesson learned: Python has alphas and betas and RCs for a reason. For future Pythons I'm going to pay attention to testing against these for my projects before the final release lands.

Reproducible Builds: Increasing the Integrity of Software Supply Chains (2021)

I didn't blog about it at the time, but a paper I co-authored with Stefano Zacchiroli was accepted by IEEE Software in April of this year. Titled Reproducible Builds: Increasing the Integrity of Software Supply Chains, the abstract of the paper is as follows:

Although it is possible to increase confidence in Free and Open Source Software (FOSS) by reviewing its source code, trusting code is not the same as trusting its executable counterparts. These are typically built and distributed by third-party vendors with severe security consequences if their supply chains are compromised.

In this paper, we present reproducible builds, an approach that can determine whether generated binaries correspond with their original source code. We first define the problem and then provide insight into the challenges of making real-world software build in a "reproducible" manner — that is, when every build generates bit-for-bit identical results. Through the experience of the Reproducible Builds project making the Debian Linux distribution reproducible, we also describe the affinity between reproducibility and quality assurance (QA).

The full text of the paper can be found in PDF format and should appear, with an alternative layout, within a forthcoming issue of the physical IEEE Software magazine.


By sil

A couple of weeks ago, my grandma died.

This was not wholly unexpected, and at the same time it was completely unexpected. I should probably explain that.

She was ninety, which is a fair age for anyone, but her mother (my great-grandmother) lived to be even older. What’s different is that nobody knew she was ninety, other than us. Most of her friends were in their mid-seventies. Now, you might be thinking, LOL, old, but this is like you in your mid-forties hanging out with someone who’s 30, or you in your late twenties hanging out with someone who’s 13, or you at eighteen hanging out with someone who’s still learning what letters are. Gaps get narrower as we get older, but the thing that most surprised me was that all her friends were themselves surprised at the age she was. She can’t have been that age, they said when we told them, and buried in there is a subtle compliment: she was like us, and we’re so much younger, and when we’re that much older we won’t be like her.

No. No, you won’t be like my grandma.

I don’t want to talk much about the last few weeks. We, my mum and me, we flew to Ireland in the middle of the night, we sorted out her house and her garden and her affairs and her funeral and her friends and her family, and we came home. All I want to say about it is that, and all I want to say about her is probably best said in the eulogy I wrote and spoke for her death, and I don’t want to say it again.

But (and this is where people in my family should tune out) I thought I’d talk about the website I made for her. Because of course I made a website. You know how some people throw themselves into work to dull the pain when something terrible happens to the people they love? I’m assuming that if you were a metalworker in 1950 and you wanted to handle your grief that a bunch of people got a bunch of metal stuff that you wouldn’t ordinarily have made. Well, I am no metalworker; I build the open web, and I perform, on conference stages or for public perception. So I made a website for my grandma; something that will maybe live on beyond her and maybe say what we thought about her.

Firstly I should say: it’s at because her name was Nell and I made it. But neither of those things are really true. Her name was Ellen, and what I did was write down what we all said and what we all did to say goodbye. I wanted to capture the words we said while they were still fresh in my memory, but more while how I felt was fresh in my memory. Because in time the cuts will become barely noticeable scars and I’ll be able to think of her not being here without stopping myself crying, and I don’t want to forget. I don’t want to lose it amongst memories of laughter and houses and lights. So I wrote down what we all said right now while I can still feel the hurt of it like fingernails, so maybe I won’t let it fade away.

I want to write some things about the web, but that’s not for this post. This post is to say: goodbye, Grandma.

Goodbye, Grandma. I tried to make a thing that would make people think of you when they looked at it. I wanted people to think of memories of you when they read it. So I made a thing of memories of you, and I spoke about memories of you, and maybe people who knew you will remember you and people who didn’t know you will learn about you from what we all said.

Goodbye, Grandma.

Abusing Terraform to Upload Static Websites to S3

Abusing Terraform to Upload Static Websites to S3

I found this really interesting. Terraform is infrastructure as code software which mostly handles creating and updating infrastructure resources, so it's a poor fit for uploading files to S3 and setting the correct Content-Type headers for them. But... in figuring out how to do that, this article taught me a ton about how Terraform works. I wonder if that's a useful general pattern? Get a tool to do something that it's poorly designed to handle and see how much you learn about that tool along the way.

Via Hacker News

Writing for distributed teams

Writing for distributed teams

Vicki Boykis describes how she only sent 11 emails during her first 12 months working at Automattic, because the company culture there revolves around asynchronous communication through durable writing using the P2 custom WordPress theme. "This is a completely different paradigm than I’ve ever worked in, which has been a world usually riddled with information lost to Slack, Confluence, and dozens of email re:re:res."

Via @vboykis

Free software activities in September 2021

Here is my monthly update covering what I have been doing in the free software world during September 2021 (previous month):


Reproducible Builds

The project is proud to be a member project of the Software Freedom Conservancy. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.

This month, I:


Elsewhere in Reproducible Builds' tooling, I made the following changes to diffoscope, including preparing and uploading versions 183, 184 and 185 and significant triaging of merge requests and other incoming issues:



Package uploads

Debian LTS

This month I have worked 18 hours on Debian Long Term Support (LTS) and 12 hours on its sister Extended LTS project.

You can find out more about the Debian LTS project in the following video:

The GIL and its effects on Python multithreading

The GIL and its effects on Python multithreading

Victor Skvortsov presents the most in-depth explanation of the Python Global Interpreter Lock I've seen anywhere. I learned a ton from reading this.

Via Hacker News

Mini Hovercraft

I challenged myself to build a mini hovercraft in only a few days as part of a new video series!

Aviation Weather Map

Sure, you could look outside for the current weather, but isn't it a lot more fun to build a live-updating map instead?

E-Paper Weather Display

What happens when you combine two-colour e-paper with bad Python? Weather! Well, weather displays.

The RFID Checklist

What do you do when you want to massively over-engineer a solution to forgetting your phone charger?

The Long Drive

Over a thousand miles over some of the loneliest areas of the USA. What's not to love?

Travel Equipment: 2019 Edition

Taking a look at some of the key travelling equipment I've grown to like in the last year of travel.

ASGI 3.0

Upgrading the ASGI spec to simplify it, while keeping backwards compatibility.

A Django Async Roadmap

Taking a look at what it would take to make Django async-native, what it enables, and if we should even do it at all.

Python & Async Simplified

Event loops, coroutines and awaits, oh my!

Channels 2.0

Finally, the promised land is here and Channels 2.0 is released. But how much has changed? And why?

CodeSOD: Read the Comments

By Remy Porter

Many many years ago, when I took my first programming course in high school, our instructor had… opinions. One of those opinions was that you should manually syntax check your code and not rely on compiler errors, because back in her day, you had to punch your program into cards, drop it off in the computer lab, wait for the technician to run your batch, and then pick up the results on the printer. It needed to work the first try or you might be waiting a whole day before you could try again.

One of her other opinions was that your code should contain as many comments as it contained lines of code. Everything needed comments. Everything. Which brings us to this code from Doug.

Private Sub BuildActuarial04_14Row(exportString As StringBuilder, record As ap_Actuarial04_14_Result) 'Card Type exportString.Append(record.CardType) 'SSN exportString.Append(record.SSN) 'Name exportString.Append(helper.FormatData(record.Name, 25, Helpers.DataType.StringFormat)) 'Sex exportString.Append(record.Sex) 'Birth Date exportString.Append(record.BirthDate) 'Hire Date exportString.Append(helper.FormatData(record.HireDate, 8, Helpers.DataType.StringFormat)) 'Plan Number exportString.Append(record.PlanNumber) 'Parish Or Other Codes 1 exportString.Append(record.ParishOrOtherCodes1) 'Parish Or Other Codes 2 exportString.Append(record.ParishOrOtherCodes2) 'Service Code exportString.Append(record.ServiceCode) 'Years Of Eligible Service exportString.Append(record.YearsOfEligibleService) 'Years Of Credited Service exportString.Append(record.YearsOfCreditedService) 'Final Average Compensation Monthly exportString.Append(record.FinalAverageCompensationMonthly) 'Annual Member Contributions For Fye exportString.Append(record.AnnualMemberContributionsForFye) 'Accumulated Member Contributions exportString.Append(record.AccumulatedMemberContributions) 'Contribution Interest exportString.Append(record.ContributionInterest) 'Termination Date exportString.Append(record.TerminationDate) 'Rehired exportString.Append(record.Rehired) 'Ibo exportString.Append(record.Ibo) 'Retirement Date exportString.Append(record.RetirementDate) 'Estimated Accrued Benefit exportString.Append(record.EstimatedAccruedBenefit) 'Original Retirement Benefit exportString.Append(record.OriginalRetirementBenefit) 'Option Code exportString.Append(record.OptionCode) 'Retirement Code exportString.Append(record.RetirementCode) 'Sex Of Beneficiary exportString.Append(record.SexOfBeneficiary) 'Beneficiary Birth Date exportString.Append(record.BeneficiaryBirthDate) 'Childs Monthly Benefit exportString.Append(record.ChildsMonthlyBenefit) 'Childs Birth Date exportString.Append(record.ChildsBirthDate) 'Years Of Military Service exportString.Append(record.YearsOfMilitaryService) 'Beneficiarys SSN exportString.Append(record.BeneficiarysSSN) 'Total Accumulated Years Of Service Purchased exportString.Append(record.TotalAccumulatedYearsOfServicePurchased) 'End Line exportString.AppendLine() End Sub

Doug's co-worker may have been in my high school class.

What's particularly ironic about this overcommenting is the method name: BuildActuarial04_14Row. What does the 04_14 mean? Why does this merit its own code path? Is that a date? A special internal code? Are there 05_14 rows?

There's a big code smell on that part, because it implies a whole suite of ap_Actuarial…Result classes that probably have the same fields, but don't have the same interface, and someone hacked together this private polymorphism.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

Error'd: Outages, Prices, and Catastrophe

By Remy Porter

Shaun F noticed an outage.

cloudflare's website demonstrating a cloudflare outage.

"Maybe," Shaun writes, "they should use the Cloudflare Always Online service."

Meanwhile, at Subway, Maurice R has found the deal of a lifetime.

A digital menu at subway where the prices are all '$null'

And as we get into Autumn, Lincoln Ramsay warns you to be on the lookout for sudden cold snaps.

A weather widget where the temperature swings from a high of 30 degrees to a low of -30 degrees
[Advertisement] Continuously monitor your servers for configuration changes, and report when there's configuration drift. Get started with Otter today!

Performance Tuning for Exabyte Queries

By Remy Porter

While NoSQL databases have definitely made their mark and have an important role in applications, there's also still a place for RDBMSes. The key advantage of an RDBMS is that, with a well normalized schema, any arbitrary query is possible, and instead of optimizing the query, you optimize the database itself to ensure you hit your performance goals- indexes, statistics, materialized views, etc..

The reality, of course, is wildly different. While the execution plan used by the database shouldn't be dependent upon how we write the query, it frequently is, managing statistics and indexes is surprisingly hard, and when performance problems crop up, without the right monitoring, it can be difficult to track down exactly which query is causing the problem.

Which brings us to this query, which TJ found while analyzing a performance problem.

select Min(the.moddate) "ModifiedDate" From T_91CDDC57 what , T_91CDDC57 the , T_91CDDC57 f where f.rdate > sysdate-1095;

First, let's just congratulate whoever named the table T_91CDDC57. I assume that's generated, and presumably so was this query. There's clearly a bug- there's no reason to have the same table in the FROM clause three times, when we just want to find the earliest moddate.

And that's the problem. T_91CDDC57 isn't a particularly large table. It's no pipsqueak- at 4.5M rows and 34M of data, it's certainly got some heft, but it's no giant, either. But that's 4.5M rows which have to be joined to 4.5M rows with no join condition, and then that gets joined to 4.5M rows again with no join condition.

Here's the explain plan of how this query executes:

----------------------------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| ----------------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 1 | 16 | 19P (1)| | 1 | SORT AGGREGATE | | 1 | 16 | | | 2 | MERGE JOIN CARTESIAN | | 18E| 15E| 19P (1)| | 3 | MERGE JOIN CARTESIAN | | 4328G| 31T| 5321M (1)| |* 4 | TABLE ACCESS FULL | T_91CDDC57 | 959K| 7499K| 18967 (2)| | 5 | BUFFER SORT | | 4509K| | 5321M (1)| | 6 | INDEX FAST FULL SCAN| T_91CDDC57_TYPE_INDEX | 4509K| | 5544 (1)| | 7 | BUFFER SORT | | 4509K| 34M| 19P (1)| | 8 | INDEX FAST FULL SCAN | T_91CDDC57_MODDATE_INDEX | 4509K| 34M| 4410 (1)| -----------------------------------------------------------------------------------------------

A few notable entries here. Line 4 does a TABLE ACCESS FULL. This is the f iteration of our table, and you can see that it pulls in just 959K rows thanks to our where clause. On line 8, you can see that it scans the T_91CDDC57_MODDATE_INDEX- it's using that index to sort so we can find the Min(the.moddate). You can also see that it touches 4509K rows. Line 6, also for 4509K rows, is our what access in the query.

Once we've accessed our three piles of data, we have to connect them. Since there's no ON or WHERE clause that links the tables, this connects each row from each table with each row in each other table. And you can see on line 3, where we join the first pair of tables, we suddenly have 4.3T rows, and a total of 31 terabytes of data. And when we join the third table it on line 2, that bloats to 18,000,000,000,000,000,000 rows and 15 exabytes of data.

TJ says:

Processing over 15 exabytes of information probably may have something to do with the performance…

Yeah, probably.

[Advertisement] Otter - Provision your servers automatically without ever needing to log-in to a command prompt. Get started today!

CodeSOD: A Ritual Approach

By Remy Porter

Frequent contributor Russell F stumbled across this block, which illustrates an impressive ability to make a wide variety of bad choices. It is written in C#, but one has the sense that the developer didn't really understand C#. Or, honestly, programming.

if (row["Appointment_Num"].ToString() == row["Appointment_Num"].ToString()) { bool b1 = String.IsNullOrEmpty(results.messages[0].error_text); if (b1 == true) { row["Message_Id"] = ($"{results.messages[0].message_id}"); row["message_count"] = ($"{results.message_count[0] }"); row["message_cost"] = ($"{results.messages[0].message_price }"); row["error_text"] = ($"{results.messages[0].error_text }"); row["network"] = ($"{results.messages[0].network }"); row["to"] = ($"{results.messages[0].to }"); row["status"] = ($"{results.messages[0].status }"); row["communicationtype_num"] = ($"1"); row["Sent_Date"] = (String.Format("{0:yyyy-MM-dd hh:mm:ss}", DateTime.Now)); row["Sent_Message"] = row["Sent_Message"].ToString(); } if (b1 == false) { row["Message_Id"] = "0000000000000000"; row["message_count"] = ($"{results.message_count[0] }"); row["message_cost"] = ($"{results.messages[0].message_price }"); row["error_text"] = ($"{results.messages[0].error_text }"); row["network"] = ($"{results.messages[0].network }"); row["to"] = ($"{results.messages[0].to }"); row["status"] = ($"{results.messages[0].status }"); row["communicationtype_num"] = ($"1"); row["Sent_Date"] = (String.Format("{0:yyyy-MM-dd hh:mm:ss}", DateTime.Now)); row["Sent_Message"] = row["Sent_Message"].ToString(); } }

Let's just start on the first line. This entire block is surrounded by a condition: row["Appointment_Num"].ToString() == row["Appointment_Num"].ToString(). If appointment num, as a string, matches appointment num, as a string, we can execute this code.

Inside of that block, we check to see if the error message IsNullOrEmpty. If it is, we'll turn it into a database row. If it's not, we'll also turn it into a database row, but with a hard-coded ID. At first glance, it's weird that they assign the IsNullOrEmpty check to a variable, but when you see that this code is written as two conditionals, instead of an if/else, you realize they forgot that else existed. They didn't want to do the IsNullOrEmpty check twice, so they stuffed it into a boolean.

It's also worth noting that the only difference between the two branches is the Message_Id, so there's a lot of duplicated code in there.

And that duplicated code is its own pile of WTFs. Clearly, the database itself is stringly-typed, as everything gets converted into a string. We mostly avoid using ToString, though, and instead use C#'s string interpolation. Which is'nt really a problem, but most of these fields are already strings. error_text, network, to, status, all strings. And `row["Sent_Message"] is a string, too- and we convert it to a string and store it in the same field.

After all that, I barely have the energy to wonder why they wrapped all of those assignments in parentheses. I suspect they misunderstood C# enough to think it's necessary. This whole thing has the vibe of "programming by incantation and ritual"- if we get the symbols in the right order and express them the right way, it will work, even if we don't understand it.

[Advertisement] Continuously monitor your servers for configuration changes, and report when there's configuration drift. Get started with Otter today!

CodeSOD: Low (Quality) Code

By Remy Porter

Like the tides, the popularity of low-code development environments comes in ebbs and flows. With each cycle, the landscape changes, old tools going away and new tools washing up on shore. One one hand, democratizing access to technology is good, on the other, these tools inevitably fail to actually do that. Instead, we get mission critical developed by people who don't understand how to develop, and are thus fragile, and released on platforms that are almost certainly going to be on legacy support in the next cycle.

I don't want to imply that low-code tools are automatically bad, or insult non-developers who want to give developing in a low-code environment a shot, though. Especially when professional developers can't really do any better.

John F's company has adopted one of the new tools in this cycle of low-code tools, Microsoft Power Apps, using the Power FX scripting language. One of the screens in their application needs to display a table of sales data, organized by month. This is how one of their developers decided to generate the list of months to draw:

          22,23,24,25,26,27,28,29], 17)

These numbers are meant to be an offset from the current month. The original developer wasn't sure how many months would be correct for their application, and wanted to experiment. This was their solution: hard-code a list that's longer than you'd reasonably want, and take the FirstN, changing N until it looks right.

Of course, "generating a sequential list of numbers of a certain length" is one of those infamous solved problems. In the case of Power FX, it's solved via the sequence function. Sequence(17, 0) would accomplish the same result.

[Advertisement] Utilize BuildMaster to release your software with confidence, at the pace your business demands. Download today!

CodeSOD: An Hourly Rate

By Remy Porter

When someone mentioned to Abraham, "Our product has an auto-sync feature that fires every hour," Abraham wasn't surprised. He was new to the team, didn't know the code well, but an auto-sync back to the server sounded reasonable.

The approach, however, left something to be desired.

syncTimer = new Timer(1000); syncTimer.Elapsed += new System.Timers.ElapsedEventHandler(syncTimer_Elapsed); syncTimer.Enabled = true; syncTimer.AutoReset = true; //... void syncTimer_Elapsed(object sender, System.Timers.ElapsedEventArgs e) { if (DateTime.Now.Second == 0 && DateTime.Now.Minute == 0) { //... do what you have to do } }

The .NET Timer object does exactly what you expect. In this case, 1,000 milliseconds after getting Enabled, it fires an event. Because AutoReset is true, it will fire that event every 1,000 milliseconds.

Now, this is supposed to trigger the auto-sync once every hour, but instead it fires once a second. On that second, we check the time- if we're on the first second of the first minute in the hour, then we fire the auto-sync operations.

There are a few problems with this. The first, and arguably most minor, is that it's just silly. Firing every second turns this timer into an "Are we there yet? Are we there yet? Are we there yet?" nuisance. It makes more sense to set the timer of 3600000 milliseconds and just have it fire once an hour.

The second problem is that this means nearly every client is going to sync at roughly exactly the same time. All of the load is going to spike all at once, at the top of the hour, every hour. That's manageable, but there's no business requirement that the sync happens at the top of the hour- any time would be fine, it should just only happen once in any given hour.

But the third problem is that this code doesn't guarantee that. This is .NET code running on Windows, and the Windows system clock can have a variety of different resolutions depending on version and configuration. In newer server versions, it could be as small as 1ms, if the sysadmin configured it for high accuracy. But this ran on clients, running desktop OSes, where the number would be closer to 15.6ms- which doesn't divide evenly into 1000ms. Which means this interval isn't going to fire exactly 1000ms apart- but 1005ms apart. Depending on what the OS is doing, it might drift even farther apart than that. But the key point is, some hours, this timer won't fire in the first second of the first minute.

The good news is that fixing this code by converting it to just fire once an hour at any time is a lot less than an hour's worth of work. Validating that in testing, on the other hand…

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

Error'd: Counting to One

By Lyle Seaman

Two of today's ticklers require a little explanation, while the others require little.

Kicking things off this week, an anonymous reporter wants to keep their password secure by not divulging their identity. It won't work, that's exactly the same as my Twitch password. "Twitch seems to be split between thinking whether my KeePass password is strong or not," they wrote. Explanation: The red translates to "This password is too easy to guess", while the green 'Stark' translates as "you've chosen a very good password indeed."



Stymied in an attempted online drugs purchase, Carl C. sums up: "Apparently a phone number is a number. I'm waiting for them to reject my credit card number because I typed it with dashes." We here at TDWTF are wondering what they'd make of a phone number like 011232110330.



Unstoppered Richard B. tops Carl's math woes, declaring "hell, after a shot of this I can't even *count* to one, let alone do advanced higher functions!"



Regular contributor Peter G. highlights "This eBay auction will ship to anywhere in the UK except the bits of the UK that are in Bolivia, Liberia, Turkmenistan, Venezuela, and a few others." Pedantically, Peter, there is no error here, unless there is in fact some bit of the UK somewhere in Sierra Leone.



Pseudonymous spelunker xyzzyl murmurs "Something tells me the rest of videos and movies is also called undefined." Don't go in there! That's a horror flick, my friend.



[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

CodeSOD: Joining the Rest of Us

By Remy Porter

Using built-in methods is good and normal, but it's certainly boring. When someone, for example, has a list of tags in an array, and calls string.Join(" ", tags), I don't really learn anything about the programmer as a person. There's no relationship or connection, no deeper understanding of them.

Which, let's be honest, is a good thing when it comes to delivering good software. But watching people reinvent built in methods is a fun way to see how their brain works. Fun for me, because I don't work with them, probably less fun for Mike, who inherited this C# code.

public List<string> Tags {get; set;} /// <summary> /// Helper function to convert a tag list to a space-delimited string representation /// </summary> /// <returns>the tags as a string separated by a space</returns> public string ToSpaceDelimitedString() { return ToDelimitedString(' '); } /// <summary> /// Helper function to convert a tag list to a delimited string representation /// </summary> /// <param name="delimiter">the delimiter to insert between the tags</param> /// <returns>the tags as a string separated by the specified delimiter</returns> private string ToDelimitedString(char delimiter) { StringBuilder delimitedTags = new StringBuilder(); foreach (string tag in Tags) { delimitedTags.AppendFormat("{0}{1}", delimitedTags.Length > 0 ? delimiter.ToString() : string.Empty, tag) ; } return delimitedTags.ToString(); }

It's important to note that ToDelimitedString is only called by ToSpaceDelimitedString, which starts us off with a lovely premature abstraction. But what I really love about this, the thing that makes me feel like I'm watching somebody's brain work, is their approach to making sure they don't have leading or training delimiters.

delimitedTags.AppendFormat("{0}{1}", delimitedTags.Length > 0 ? delimiter.ToString() : string.Empty, tag)

On the first run of the loop, delimitedTags is empty, so we append string.Empty, tag- so just tag. Every other iteration of the loop, we append the delimiter character. I've seen lots of versions of solving this problem, but I've never seen this specific approach. It's clever. It's not good, but it's clever.

And, as is good practice, it's got a unit test:

[Test] public void ToSpaceDelimitedString() { TagList list = new TagList(_blogKey); string expected = "tag1 tag2 tag3"; foreach (string tag in expected.Split(' ')) { list.Add(tag); } string actual = list.ToSpaceDelimitedString(); Assert.AreEqual(expected, actual, "ToSpaceDelimitedString failed"); }

What's interesting here is that they know about string.Split, but not string.Join. They're so close to understanding none of this code was needed, but still just a little too far away.

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

CodeSOD: Supporting Standards

By Remy Porter

Starting in the late 2000s, smartphones and tablets took off, and for a lot of people, they constituted a full replacement for a computer. By the time the iPad and Microsoft Surface took off, every pointy-haired-boss wanted to bring a tablet into their meetings, and do as much work as possible on that tablet.

Well, nearly every PHB. Lutz worked for a company where management was absolutely convinced that tablets, smartphones, and frankly, anything smaller than the cheapest Dell laptop with the chunkiest plastic case was nothing more than a toy. It was part of the entire management culture, led by the CEO, Barry. When one of Lutz's co-workers was careless enough to mention in passing an article they'd read on mobile-first development, Barry scowled and said "We are a professional software company that develops professional business software."

Back in the mid 2010s, their customers started asking, "We love your application, but we'd love to be able to access it from our mobile devices," Barry's reply was: "We should support standards. The standard is Microsoft Windows."

"Oh, but we already access your application on our mobile devices," one of the customers pointed out. "We just have to use the desktop version of the page, which isn't great on a small screen."

Barry was livid. He couldn't take it out on his customers, not as much as he wanted to, but he could "fix" this. So he went to one of his professional software developers, at his professional software company, and asked them to professionally add the following check to their professional business software:

Public Sub OnActionExecuting(filterContext As ActionExecutingContext) Implements IActionFilter.OnActionExecuting Dim userAgent As String = filterContext.HttpContext.Request.UserAgent If Not userAgent.Contains("Windows NT") OrElse userAgent.Contains("; ARM;") Then filterContext.Result = New ContentResult With {.Content = "Your operating system is not supported. Please use Microsoft Windows."} End If End Sub

Filtering users based on User-Agent strings is a bad idea in general, requiring it to contain "Windows NT" is foolish, but banning UA-strings which contain "ARM" is pure spite. It was added specifically to block, at the time, Windows RT- the version of Windows built for the Surface tablet.

There's no word from Lutz about which lasted longer: this ill-conceived restriction or the company itself.

[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

CodeSOD: Like a Tree, and…

By Remy Porter

Duncan B was contracting with a company, and the contract had, up to this point, gone extremely well. The last task Duncan needed to spec out was incorporating employee leave/absences into the monthly timesheets.

"Hey, can I get some test data?" he asked the payroll system administrators.

"Sure," they said. "No problem."

{ "client": "QUX", "comp": "FOO1", "employee": "000666", "employeename": { "empname": "GOLDFISH, Bob MR", "style": "bold" }, "groupname": "manager GOLDFISH, Bob MR", "drillkey": { "empcode": { "companyid": "FOO1", "employeeid": "000666" } }, "empleaves": { "empleave": [ { "empcode": { "companyid": "FOO1", "employeeid": "000333" }, "name": "AARDVARK, Alice MS", "candrill": 0, "shortname": "AARDVARK, Alice", "subposition": "", "subpositiontitle": "", "leavedays": { "day": [ "","","","","","","","","AL","","","","","", "","","","","","","","","","","","","","", "","","","" ] } }, { "empcode": { "companyid": "FOO1", "employeeid": "000335" }, "name": "AARDWOLF, Aaron MR", "candrill": 0, "shortname": "AARDWOLF, Aaron", "subposition": "", "subpositiontitle": "", "leavedays": { "day": [ "","","","","","","","","","","","","","", "","","","","","","","","","","","","","", "","" ] } } ] } } }

Well, there were a few problems. The first of which was that the admins could provide test data, but they couldn't provide any documentation. It was, of course, the leavedays field which was the most puzzling for Duncan. On the surface, it seems like it should be a list of business days within the requested range. If an employee was absent one day, it would get marked with a tag, like "AL", presumably shorthand for "allowed" or similar.

But that didn't explain why "AARDWOLF Aaron" had fewer days that "AARDVARK Alice". Did the list of strings somehow tie back to whether the employee were scheduled to work on a given day? Did it tie to some sort of management action? Duncan was hopeful that the days lined up with the requested range in a meaningful way, but without documentation, it was just guessing.

For Duncan, this was… good enough. He just needed to count the non-empty strings to drive his timesheets. But he feared for any other developer that might want to someday consume this data.

Duncan also draws our attention to their manager, "GOLDFISH, Bob MR", and the "style" tag:

I'm fairly sure that's a hint to the UI layer, rather than commentary on the Mr. Goldfish's management style.

[Advertisement] Continuously monitor your servers for configuration changes, and report when there's configuration drift. Get started with Otter today!

CodeSOD: Price Conversions

By Remy Porter

Russell F has an object that needs to display prices. Notably, this view object only ever displays a price, it never does arithmetic on it. Specifically, it displays the prices for tires, which adds a notable challenge to the application: not every car uses the same tires on the front and rear axles. This is known as a "staggered fitment", and in those cases the price for the front tires and the rear tires will be different.

The C# method which handles some of this display takes the price of the front tires and displays it quite simply:

sTotalPriceT1 = decTotalPriceF.ToString("N2");

Take the decTotalPriceF and convert it to a string using the N2 format- which is a number with thousands separators and two digits behind the decimal place. So this demonstrates that the developer responsible for this code understands how to format numbers into strings.

Which is why it's odd when, a few lines later, they do this, for the rear tires:

sTotalPriceT2 = decimal.Parse(decTotalPriceR.ToString("F2")).ToString("N2");

We take the price, convert it to a string without thousands separators, then parse it back into a decimal and then convert it to a string with thousands separators.

Why? Alone, this line would just be mildly irksome, but when it's only a few lines below a line which doesn't have this kind of ridiculousness in it, the line just becomes puzzling.

But the puzzle doesn't end. sTotalPriceT1 and sTotalPriceT2 are both string variables that store the price we're going to display. Because this price information may need to be retained across requests, though, someone decided that the prices also need to get stored in a session variable. In another method in the same class:

Session["FOS_TPriceF"] = bStaggered ? decimal.Parse(sTotalPriceT1).ToString("N2") : null; Session["FOS_TPriceR"] = bStaggered ? decimal.Parse(sTotalPriceT2).ToString("N2") : null;

Once again, we're taking a string in a known format, turning it back into the base numeric type, then formatting back to the format it already was. And I suppose it's possible that some other bit of code may have modified the instance variables sTotalPriceTN and broken the formatting, but it seems to me the solution is to not store numbers as strings and just format them at the moment of display.

[Advertisement] BuildMaster allows you to create a self-service release management platform that allows different teams to manage their applications. Explore how!

Error'd: Money for Nothin'

By Lyle Seaman

... and gigs for free.

"Apple is magical," rhapsodizes music-lover Daniel W.



Meanwhile, in the overcast business district of Jassans-Riottier, where the stores are all closed but there's a bustle in the hedgerow, local resident Romain belts "I found the Stairway to Heaven just 100m from my house!"



Yes, there are two paths you can go by.



But in the long run ... you won't get there on any of these buses, shared by Alex Allan, wailing "you wait for one, and XXX all come together!"



Nor on this train of the damned from Finn Jere


No, it's very clear the mandatory mode is Zeppelin.

~ ~
[Advertisement] Keep the plebs out of prod. Restrict NuGet feed privileges with ProGet. Learn more.

CodeSOD: Making Newlines

By Remy Porter

I recently started a new C++ project. As it's fairly small and has few dependencies, I made a very conscious choice to just write a shell script to handle compilation. Yes, a Makefile would be "better", but it also adds a lot of complexity my project doesn't need, when I can have essentially a one-line build command. Still, my code has suddenly discovered the need for a second target, and I'll probably migrate to Makefiles- it's easier to add complexity when I need it.

Kai's organization transitioned from the small shell-scripts approach to builds to using Makefiles about a year ago. Kai wasn't involved in that initial process, but has since needed to make some modifications to the Makefiles. In this case, there's a separate Makefile for each one of their hundreds of microservices.

Each one of those files, near the top, has this:

# Please note two empty lines, do not change define newline endef

The first time Kai encountered this, a CTRL+F showed that the newline define was never used. The second time, newline was still unused. Eventually, Kai tracked back to one of the first Makefiles, and found a case where it was actually used. Twice.

It was easy to understand what happened: someone was writing a new Makefile, and looked at an older one for an example, and probably copy/pasted a lot of it. They saw a comment "do not change", and took this to mean that they needed to include this for reasons they didn't understand. And now, every Makefile has this do-nothing define for no real reason.

Kay writes:

Since finding out about this, I kept wondering what would happen if I started adding ASCII cows into those files with the comment: "Please note this cow, do not change"

_____________________________________ < Please note this cow, do not change > ------------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || ||
[Advertisement] Otter - Provision your servers automatically without ever needing to log-in to a command prompt. Get started today!

Editor's Soapbox: Eff Up Like It's Your Job

By Remy Porter

This past Monday, Facebook experienced an outage which lasted almost six hours. This had rattle-on effects. Facebook's pile of services all failed, from the core application to WhatsApp to Oculus. Many other services use Facebook for authentication, so people lost access to those (which highlights some rather horrifying dependencies on Facebook's infrastructure). DNS servers were also strained as users and applications kept trying to find Facebook, and kept failing.

CloudFlare has more information about what went wrong, but at its core: Facebook's network stopped advertising the routes to its DNS servers. The underlying cause of that may have been a bug in their Border Gateway Protocol automation system:

How could a company of Facebook’s scale get BGP wrong? An early candidate is that aforementioned peering automation gone bad. The astoundingly profitable internet giant hailed the software as a triumph because it saved a single network administrator over eight hours of work each week.
Facebook employs more than 60,000 people. If a change designed to save one of them a day a week has indeed taken the company offline for six or more hours, that's quite something.

Now, that's just speculation, but there's one thing that's not speculation: someone effed up.

IT in general, and software in specific, is a rather bizarre field in terms of how skills work. If, for example, you wanted to get good at basketball, you might practice free-throws. As you practice, you'd expect the number of free-throws you make to gradually increase. It'll never be 100%, but the error rate will decline, the success rate will increase. Big-name players can expect a 90% success rate, and on average a professional player can expect about an 80% success rate, at least according to this article. I don't actually know anything about basketball.

But my ignorance aside, I want you to imagine writing a non-trivial block of code and having it compile, run, and pass its tests on the first try. Now, imagine doing that 80% of the time.

A meme of a man staring at a computer, puzzled, repeated twice. First, he thinks: 'My code doesn't work, I have no idea why.'. Second, he thinks: 'My code works, I have no idea why.

It's a joke in our industry, right? It's a joke that's so overplayed that perhaps it should join "It's hard to exit VIM" in the bin of jokes that needs a break. But why is this experience so universal? Why do we have a moment of panic when our code just works the first time, and we wonder what we screwed up?

It's because we already know the truth of software development: effing up is actually your job.

You absolutely don't get a choice. Effing up is your job. You're going to watch your program crash. You're going to make a simple change and watch all the tests go from green to red. That semicolon you forgot is going to break the build. And you will stare at one line of code for six hours, silently screaming, WHY DON'T YOU WORK?

And that's because programming is hard. It's not one skill, it's this whole complex of vaguely related skills involving language, logic, abstract reasoning, and so many more cognitive skills I can't even name. We're making thousands of choices, all the time, and it's impossible to do this without effing up.

Athletes and musicians and pretty much everybody else practices repeating the same tasks over and over again, to cut down on how often they eff up. The very nature of our job is that we rarely do exactly the same task- if you're doing the same task over and over again, you'd automate it- and thus we never cut down on our mistakes.

Your job is to eff up.

You can't avoid it. And when something goes wrong, you're stuck with the consequences. Often, those consequences are just confusion, frustration, and wasted time, but sometimes it's much worse than that. A botched release can ruin a product's reputation. You could take down Facebook. In the worst case, you could kill someone.

But wait, if our job is to eff up, and those mistakes have consequences, are we trapped in a hopeless cycle? Are we trapped in an existential crisis where nothing we do has meaning, god is dead, and technology was a mistake?

No. Because here's the secret to being a good developer:

You gotta get good at effing up.

The difference between a novice developer and an experienced one is how quickly and efficiently they screw up. You need to eff up in ways that are obvious and have minimal consequences. You need tools, processes, and procedures that highlight your mistakes.

Take continuous integration, for example. While your tests aren't going to be perfect, if you've effed up, it's going to make it easier to find that mistake before anybody else does. Code linting standards and code reviews- these are tools that are designed to help spot eff ups. Even issue tracking on your projects and knowledge bases are all about remembering the ways we effed up in the past so we can avoid them in the future.

Your job is to eff up.

When looking at tooling, when looking at practices, when looking at things like network automation (if that truly is what caused the Facebook outage), our natural instinct is to think about the features they offer, the pain points they eliminate, and how they're better than the thing we're using right now. And that's useful to think about, but I would argue that thinking about something else is just as important: How does this help me eff up faster and more efficiently?

New framework's getting good buzz? New Agile methodology promises to make standups less painful? You heard about a new thing they're doing at Google and wonder if you should do it at your company? Ask yourself these questions:

Your job is to eff up.

The more mistakes you make, the better a programmer you are. Embrace those mistakes. Breaking the build doesn't make you an imposter. Spending a morning trying to track down a syntax error that should be obvious but you can't spot it for the life of you doesn't mean you're a failure as a programmer. Shipping a bug is inevitable.

Effing up is the job, and those eff ups aren't impediments, but your stepping stones. The more mistakes you make, the better you'll get at spotting them, at containing the fallout, and at learning from the next round of mistakes you're bound to make.

Now, get out there and eff up. But try not to take down Facebook while you do it.

[Advertisement] Otter - Provision your servers automatically without ever needing to log-in to a command prompt. Get started today!

CodeSOD: Unzipped

By Remy Porter

When you promise to deliver a certain level of service, you need to live up to that promise. When your system is critical to your customers, there are penalties for failing to live up to that standard. For the mission-critical application Rich D supports, that penalty is $10,000 a minute for any outages.

Now, one might think that such a mission critical system has a focus on testing, code quality, and stability. You probably don't think that, but someone might expect that.

This Java application contains a component which needs to take a zip file, extract an executable script from it, and then execute that script. The code that does this is… a lot, so we're going to take it in chunks. Let's start by looking at the core loop.

private void extractAndLaunch(File file, String fileToLaunch) { try { ZipInputStream zipIn = new ZipInputStream(new FileInputStream(file)); ZipEntry entry = zipIn.getNextEntry(); byte[] buffer = new byte[1024]; while (entry != null) { if (!entry.isDirectory() && entry.getName().compareToIgnoreCase(fileToLaunch) == 0) { // SNIP, for now } } zipIn.closeEntry(); entry = zipIn.getNextEntry(); zipIn.close(); } catch (Exception e) { LOG.error("Failed to load staging file {}", file, e); } }

So, we create a ZipInputStream to cycle through the zip file, and then get the first entry from it. While entry != null, we do a test: if the entry isn't a directory, and the name of the entry is the file we want to launch, we'll do all the magic of launching the executable. Otherwise, we go back to the top of the loop, and repeat the same check, on the same entry, forever. If the first file in this zip file isn't the file we want to execute, this falls into an infinite loop, because the code for cycling to the next entry is outside of the loop.

This may be a good time to point out that this code has been in production for four years.

Okay, so how do we extract the file?

// Extract the file File newFile = new File(top + File.separator + entry.getName()); LOG.debug("Extracting and launching {}", newFile.getPath()); // Create any necessary directories try { new File(newFile.getParent()).mkdirs(); } catch (NullPointerException ignore) { // CS:NO IllegalCatch // No parent directory so dont worry about it } int len; FileOutputStream fos = new FileOutputStream(newFile); while ((len = > 0) { fos.write(buffer, 0, len); } fos.close(); zipIn.closeEntry();

So, first, we build a file path with string concatenation, which is ugly and avoidable. At least they use File.separator, instead of hard-coding a "/" or "\". But there's a problem with this: top comes from a configuration file and is loaded by System.getProperty(), which may not be set, or may be an empty string. This means we might jam things into a directory called null, or worse, try and extract to the root of the filesystem.

Which also means that newFile.getParent() may be null. Instead of checking that, we'll just catch any exceptions it throws.

We also call zipIn.closeEntry() here, and we close the same entry again after the loop. I assume the double close doesn't hurt anything, but it's definitely annoying.

Okay, so how do we execute the file?

OsHelper.execute(newFile.getName(), newFile.getParentFile()); // Execute the file // TODO Fix this to work under linux List<String> cmdAndArgs = Arrays.asList("cmd", "/c", fileToLaunch); ProcessBuilder pb = new ProcessBuilder(cmdAndArgs); File(System.getProperty("top"))); Process p = pb.start(); InputStream error = p.getErrorStream(); byte[] errBuf = new byte[1024]; if (error.available() > 0) {; LOG.error("Script {} had error {}", fileToLaunch, errBuf); } int exitValue = 0; while (true) { try { exitValue = p.exitValue(); break; } catch (IllegalThreadStateException ignore) {} // Just waiting for the batch to end }"Script " + fileToLaunch + " exited with status of " + exitValue); newFile.delete(); break;

OsHelper.execute does not, as the name implies, execute the program we want to run. It actually sets the executable bit on Linux systems. It doesn't use any Java APIs to do this, but just calls chmod to mark the file as executable.

Of course, that doesn't matter, because as the comment explains: this doesn't actually work on Linux. They actually shell out to cmd to run it, the Windows shell.

Then we launch the script, running it in the working directory specified by top, but instead of re-using the variable, we fetch it from the configuration again. We read from standard error on the process, but we don't wait, so most of the time this won't give us anything. We'd have to be very lucky to get any output from this running process.

Then, we wait for the script to complete. Now, it's worth noting that there's a Java built-in for this, Process#waitFor() which will idle until the process completes. Idle, instead of busy wait, which is what this code does. It's also worth noting that Process#exitValue() throws an exception if the process is still running, so in practice this code spams IllegalThreadStateExceptions as fast as it can.

Finally, none of these exception handlers have finally blocks, so if we do get an error that bubbles up, we'll never call newFile.delete(), leaving our intermediately processed work sitting there.

The code, in its entirety:

private void extractAndLaunch(File file, String fileToLaunch) { try { ZipInputStream zipIn = new ZipInputStream(new FileInputStream(file)); ZipEntry entry = zipIn.getNextEntry(); byte[] buffer = new byte[1024]; while (entry != null) { if (!entry.isDirectory() && entry.getName().compareToIgnoreCase(fileToLaunch) == 0) { // Extract the file File newFile = new File(top + File.separator + entry.getName()); LOG.debug("Extracting and launching {}", newFile.getPath()); // Create any necessary directories try { new File(newFile.getParent()).mkdirs(); } catch (NullPointerException ignore) { // CS:NO IllegalCatch // No parent directory so dont worry about it } int len; FileOutputStream fos = new FileOutputStream(newFile); while ((len = > 0) { fos.write(buffer, 0, len); } fos.close(); zipIn.closeEntry(); OsHelper.execute(newFile.getName(), newFile.getParentFile()); // Execute the file // TODO Fix this to work under linux List<String> cmdAndArgs = Arrays.asList("cmd", "/c", fileToLaunch); ProcessBuilder pb = new ProcessBuilder(cmdAndArgs); File(System.getProperty("censored"))); Process p = pb.start(); InputStream error = p.getErrorStream(); byte[] errBuf = new byte[1024]; if (error.available() > 0) {; LOG.error("Script {} had error {}", fileToLaunch, errBuf); } int exitValue = 0; while (true) { try { exitValue = p.exitValue(); break; } catch (IllegalThreadStateException ignore) {} // Just waiting for the batch to end }"Script " + fileToLaunch + " exited with status of " + exitValue); newFile.delete(); break; } } zipIn.closeEntry(); entry = zipIn.getNextEntry(); zipIn.close(); } catch (Exception e) { LOG.error("Failed to load staging file {}", file, e); } }
[Advertisement] ProGet’s got you covered with security and access controls on your NuGet feeds. Learn more.

A quiet October

By [email protected] (Jon North)

After the short stormy interlude, autumn is drifting in with sunshine across the garden, late flowers (these are pomegranate or grenadier) and ripening fruits (these from our always productive kaki or persimmon - the tree is known as a plaqumenier).  Although things are becoming more active in our lives - choir reorganisation and Anglophone library stockchecking on the horizon for me, and lots of cello practice for Mary - it is a tranquil time, so I have mainly photos to share this time on the blog.

Canalside walks are regular pleasures in the sunshine, reflections from the water, and various exotic plants along the streets.

And when activities are over, there are always messages to catch up with, and things to watch and listen to on tv and podcasts.  Elvire and Edmond in attendance!

It never rains but it pours

By [email protected] (Jon North)

Tuesday's very local storm - we were driving towards Calvisson at the time!!

I started this post last Tuesday morning, a couple of hours after one of the most spectacular storms I can remember.   We had set out for our Tuesday conversation group in Congénies, about 20 km away, and it had just started to rain, but thunder and forked lightning were soon overhead.  By the time we'd travelled over halfway the heavens opened even more and we were forced to stop, with rain lashed by high winds and bits of ice banging the car.  After about a quarter of an hour we decided to turn back and barely made it past bad flooding in several dips in the road, including back in the centre of Lunel, but as we arrived home the rain had all but stopped.  The power was off but thankfully just via the house trip switch and now everything is back on and calm reigns.  Others who had hoped to join us for the morning set apologies, luckily having cancelled their plans sooner.

It was the beginning of a few days of wet weather - 95 mm of rain all told - but above all (seeing the reports of hundreds stranded in their cars and washed sideways by the floods very close to our point of turning round) we feel very lucky to have escaped any bad consequences.  Now the sun is shining again.  We are also lucky to live surrounded by large gardens so that water mostly drains away quickly.  This area is notorious for severe floods from the rivers that flow down from the Cevennes, but the rain that falls actually on us causes problems for a few hours then sinks away.  No thanks to drains in the past - generally they have not been a high priority in towns around here because rain is relatively rare - but more recently they have started to install adequate drains.

We were glad to be back safely because Mary had her latest eye operation that afternoon.  After 2 successful cataract ops, this one was to lift her eyelids which had begun to hood her vision a little.  She thought this might count as cosmetic, but he surgeon assured her that is was medical need and so can be covered by the usual health service provisions.  The procedure was successful and, once the bruising fades, will be a real improvement.  Once again we are grateful and impressed at the quality of health care here.

We had just returned from a wonderful birthday trip to Burgundy.  Of course this was partly about wine, and probably will feature in the wine blog in due course.  We stayed a few nights in a really nice hotel in Vougeot, visited Beaune, saw winemakers we know well and had an impromptu drive across a very rural part of the countryside away from the vines after a friend we'd hoped to see was in hospital for a heart emergency (she's thankfully recovered now).  She lives near Châteauneuf, NW of Beaune, pictured here.  Altogether an enjoyable trip. 

La rentrée chez North

By [email protected] (Jon North)

September, and the internet in France is as ever full of ads for school goods - French students are obliged to stock up on a whole list of supplies, and stationery businesses and supermarkets alike want you to believe that theirs are the best pencils, calculators, school bags or whatever at reasonable prices.  For us older folk, it is a time when all sorts of activities restart - in the next three weeks my choir, Mary's chamber music and cello lessons and our SEVE network (which includes our Tuesday language groups) all restart and the Anglophone Library in Montpellier reopens.  And we need to check our diaries more often to avoid missing appointments in a still uncrowded week.

An old library layout now being changed
The latter is now under new management after the kind of enforced committee reshuffle I was familiar with in England - old hands and practices became creaky and cracks showed in organisation.  Inevitably (as a retired librarian among other things) I have been drafted in to help, but it is a pleasure - I have already staffed the library on a few of its twice weekly openings - and there is a big job to do in reviving and broadening the book selection process.  These days I do most of my reading on the Kindle, but I still have an affection for physical books.

This blog has often been about health.  As I write Mary is at the ophthalmologists for a checkup following two apparently successful cataract operations.  When cleared she has promised herself new glasses, though her consultant says the prescription has not changed much.  I meanwhile am on a plateau of manageable pain with my new doctor (a 4th in a year following our surviving Dr Cayla's retirement) trying to persuade me that too much painkiller is poisonous.  I don't really need convincing, and it's quite interesting to find out how little gradual reductions in doses changes things.

Twin dogs Edmond and Elvire have just celebrated their 12th birthdays.  They are in good spirits and health, now that we have sorted out Elvire's bladder problem - I need to take her for a quick pee in the middle of the night, which is really no problem since I wake often anyway and get back to sleep easily enough after the brief foray.  I'm gradually returning to normal after I recovered from my broken arm at the beginning of the year.

We had a nice moment today. I was anxious over the past week because our 🍷 cooling system had broken down and over 200 bottles were (still are) at risk in this hot weather. I could not for the life of me remember who had repaired it last time - it ran faultlessly for over 9 years then, after repair, another 5+. Anyway, Mary phoned Languedoc Depannage, who mended our dishwasher a few months since, and lo, it was them! The very nice lady instantly knew about the last repair and understanding (as people down here do) the dangers of cooling failure for wine stores, is sending one of her nice young men to look this afternoon, estimate tomorrow, repairs on the horizon. Service is not dead!

Something lighthearted to end with. We are off to celebrate our own birthdays in Burgundy for a few days!


New discoveries and old friends

By Jon North ([email protected])

At the Mas de Bellevue above Lunel to the north

I've nearly begun this post several times over the summer.  I do so now in memory of my recently deceased friend Alan Byars, married to Mary's cousin Barbara.  Although he made his money from another liquid, oil, this larger than life Texan shared my interest in wine and encouraged me to revive this blog when we met over good glasses whether in New Mexico or the Rhône valley.  He was an atypical American, (many I know don't even have passports) having lived and worked in Europe as well as in the US, and he and Barbara shared many holidays with us in England and in France as well as America.

Last time I wrote of our winter excursions into Spanish wines, and we have continued to enjoy the fruits of  our discovery of the Barcelona wine merchant Decántalo whose deliveries and service are very good.  Now I want to write a little about more local wines, which of course have been at the heart of our enjoyment of wine since our first excursions to France in the 1990s.   Our most recent discovery has been a wine truck, a motorised market stall that turns up at markets and evening events in Lunel and other local towns.  It's based near the Pic Saint Loup, one of our best-known local appellations, which can be seen (as here from the Mas de Bellevue) on the horizon from many places around here, and it's called Dégustez Sud, run by a nice couple who spend a lot of time picking out good wines from small producers across the area from Spanish border to the Rhône.  The wines are not necessarily cheap, but this is largely because smaller producers have higher overheads than larger-scale winemakers.

So far we have tried two whites, Folio, a grenache gris from Collioure near the Spanish border and a Mas d'Amile terret blanc, neither cheap but both out of the ordinary and enjoyable.  White wines can be more expensive than run-of-the-mill reds despite the generally shorter making time, because the process has to be cleaner and more temperature controlled.

Pleasant evening outings into Lunel to try wines from Dégustez Sud, the second with oysters for Mary!

Terret used to be a grape only used for fortified apéritif wines like Noilly Prat, but recently some delicious dry white wines have been made from this grape, and one from the Domaine de la Fadèze near Mèze has long been one of our favourites.  Interestingly, in a comparative tasting of that and the Amile wine the other day, we both preferred the cleaner, lemony Fadèze to the more recent discovery, and it has the benefit of being less than a third of the price, but that may be partly because we go to the producers for the Fadèze.  They make a range of red, white and rosé single-variety wines, all at very good prices.  

I'm ending with a few of the labels of recently enjoyed wines - 2 from the Rhône valley, a 10-year-old Beaumes de Venise from the left bank of the Rhône, in perfect condition from Durban, one of the first producers we visited there and the other from the right bank, the newere additions to the Rhône area in the Gard, in fact near the Pont du Gard and so named Domaine de l'Aqueduc.   And 2 from the Rive droite of the Garonne, one of the good sweet wines opposite the great Sauternes châteaux, and a very good Entre Deux Mers white from our good friend Jérôme whom we met recently in the Drôme, another proof that good white wines can keep for several years.



Les vacances continuent

By [email protected] (Jon North)


in the Drôme near Crest

Last week we went on holiday.  Since we can only now start planning trips to England, our current breaks are in France, and this trip was to the area we know well on the other side of the Rhône.  In fact it was the twinning link with Die that pulled us towards France from 1992 onwards, and over more than 10 years we visited the area 2 or 3 times a year.  We made friends there, and gradually got to know the neighbouring areas of Provence to the south and then the Languedoc further west, where we eventually decided to live.

Die is mountainous, and we prefer the flatter lands to live in, but we love revisiting our old haunts and discovering new bits - this was our second visit to the Crest area, between Die and Valence.  We found a simple comfortable gîte and drove around to Die, Châtillon-en-Diois and Saint Andéol-en-Quint (also an early holiday place for us, the first place we took Sam to, and where other ex-Wirksworth friends have now settled), and combined some gentle tourism and a bit of wine buying with visits to several good friends.

The countryside south of Crest and towards Saillans is dominated by an extraordinary mountainous area, a kind of long oval with forest in the centre, the forêt de Saou.  The mountains culminate at their eastern end by the Trois Becs, peaks visible from the A7 motorway as you approach from the south.  Jérôme, whom we welcomed to stay with us in Wirksworth in the 1990s and who now lives and works in Germany, invited us to eat with him in the little village of Saou, only 10 minutes from our gîte, which has a slightly alternative air and was humming with post-covid enjoyment the evening we ate their with him and his son Oscar.  It's a centre for walking and climbing, and Jérôme has known the area well since his youth growing up in Châtillon.

We were delighted to be able to visit Jérôme's parents Pierre and Michelle during our holiday.  Pierre was the percepteur (tax official) in Châtillon while Michelle was a teacher - over the years we stayed several times in their flat over the trésorerie and, when they retired, they built a house in the village righ on the little market square, the Champ de Foire.  they had both had covid but luckily were not too seriously ill.

We spent a morning wandering round Crest and had lunch in a pleasant restaurant there.  The tower, a former prison, is a landmark in the area, and we also visited Die itself, and spent some time with Krys, one of our oldest twinning friends.  On our final evening we also visited old Wirksworth friends Clare and Alan who moved to the area permanently after retirement - a very good evening meal in their garden in St Andéol, but the storm that accompanied our drive back to the gîte was not so much fun!  The links forged by twinning have stood the test of time.

the 7 deadly sins carved on a building in Crest - sloth, pride, envy, wrath, avarice, gluttony and lust

The Tour ends and the blog returns

By [email protected] (Jon North)

Photos mostly of skies and garden to accompany this blog post

I'm glad to sit down and write again after some weeks taken up with following a fascinating Tour de France.  The news is full of Covid still of course , but now mingled with every louder cries of freedom, of liberté and of the irrepressible French need to protest, so that although France is much more controlled than the UK Macron, is seen as a dictator denying people their rights.  The latest manifestation of the gilet jaune mentality - I prefer the cycling version.

Our view of 'getting back to normal' is coloured by ever-closer knowledge of the dangers of the pandemic, most recently when Mary met our friends  Daniel & Josette in the street last week.  They were among the first French friends we made through casual meeting at a musical event soon after we arrived, and both have had their share of health problems, but recently Daniel came close to death through Covid, and it is a great blessing that he recovered.

Vaccination was a concern for us in the late spring, but thanks to help from our doctors' practice we both had appointments and double doses fairly promptly and now have all the QR coded documents to confirm it.  We are now also legitimate residents of la belle France.  My permit arrived by post soon after our February interviews at the Préfecture, but we are grateful to another French friend (again since the early days of our arrival here): he works for a mairie, and Mary was convinced that council officers did not have to flounder in the labyrinth of internet contacts and phone queues to be able to phone up and chase her application.  He told us afterwards that it had taken even him several attempts to get through to the right person, but within a week or of his calls Mary had not one but two appointments at the Préfecture in Montpellier, and now has her card.

On the home front I've sold my bike, which gave me over 10 years of enjoyment.  I still enjoy riding in
theory, but feel a little uncertain of my balance since my accident so have decided to get my exercise by walking or daily sessions on the static bike I have.  A pity though that I'll not get to ride the two new cycle routes from Lunel, to Marsillargues and to La Grande Motte - maybe I'll hire a bike sometime and ride them.  That and the cross trainer I also use most days have confirmed their excellent value compared with gym subs, especially since they are on site and I can choose my own music or podcasts.  Podcasts are a never-ending source of interest and variety - I'll do a round-up of our favourites during the summer.

The man who bought my bike is our gardener and handyman, M. Beaumann, introduced to us by friend Dawn who lives the other side of Lunel, who has just built our new shed to replace the ramshackle one I put up soon after we arrived.  He cleared the bottom of the garden and put up a new fence too, and although we wondered if we'd lose privacy, in practice it has been a real improvement to the garden, and an old vine and bamboo in pots have rapidly made the boundary green again.  I enjoy the garden at night as well as during the day because I need to take Elvire out for a pee in the early hours, and on one occasion she took a time to perform.  One of the neighbours must have seen me and called the police in case I and was an intruder - when I got back inside I saw blue flashing lights cruising in front of the house, but they must have had a good look and decided I belonged here - they just drove off and we heard no more.

Our Tuesday language groups are starting again - we are hosting tomorrow - and it's quite strange getting back into a rhythm of meeting and learning, but we really enjoy seeing our friends French, English and American plus some other Europeans!

In a week or so we are looking forward to a holiday in the Drôme, near our old twin area of the Diois - a gîte where dogs are allowed, and our first time away since we visited Nigel & Elizabeth.  Of course we still have no idea when we'll be able to travel to see our family and friends in the UK.  We're just beginning to plan other trips around France, not unconnected with wine!

By [email protected] (Jon North)

After a beautifully sunny start to June, his week in Lunel has turned muggy and grey.  Little rain though, despite rain and storms all round us.

The blog is back. Not really a break but a lazy delay in putting together this next post.  That means there are plenty of pictures to share, and the news here is that we have a new garden shed - our friendly workman M. Beaumann is just finishing the concrete floor as I write.

The first pic shows the old and very ramshackle shed, which lasted nearly 15 years.  You can see the tortoise in front of her little shelter in the second photo

Last week we paid a welcome visit to our friends Nigel & Elizabeth near Narbonne.  Four sets of photos, one of the river Aude near their house, seen during our dog walks, and three others at Narbo Via, the new and impressive Norman Foster museum on the outskirts of Narbonne.  This town was the commercial centre of the Roman empire in France, but without the monumental buildings of Nîmes or Arles.  So the remains had to be reassembled from the later buildings into which they'd been incorporated, and the main display is a kind of giant library of stone carvings.  Foster, we think, has done France proud!

This summer the town has come to life a bit more with brightly coloured animal sculptures

Finally the garden continues to provide colour and interest - this time from lilies and (ornamental) pomegranates as well as ever-changing light on our own sculptures.

A month of holidays

By [email protected] (Jon North)

I begin this blog with a boast about our wonderful daughter-in-law Fi, who has just been awarded an honorary doctorate by Keele University.  Here's the announcement, excellent news for the theatre too because her colleague Theresa also received an award as you can see.  The New Vic has just reopened after a long layoff - in France too, culture and arts are tentatively coming back to life, and choral singing is not lagging behind - we hear that in England it has still been singled out as especially risky

May in France is, it seems, stuffed with holidays.  It is not the first time I've commented that many of these are Catholica, Christian, religious festivals - as I write this Monday afternoon we have the Pentecôte holiday, what in England would be called Whit Monday.  Years ago when our new library, the médiatheèque, opened to fanfares (rightly) and soon afterwards the town announced that it would be open on the first Sunday of each month.  That year, the first one, the first Sunday of April was April, was Easter Sunday and - guess what - the Library was closed!  France will continue to honour its Catholic heritage but inconsistently refuses to acknowledge equally other religions such as Islam.

I was intrigued this week by this piece by Duncan Campbell about Bow Street on its final day as a functioning court.  On that day in 2006 he was a member of the press bench  "I still have my pass). A young woman, arrested for posting advertisements for sex workers in telephone boxes, was given a conditional discharge and ordered to surrender her Blu Tack. A businessman facing extradition to Germany for a £14 million fraud case was remanded in custody. Two Kosovans were accused of paying £4000 to buy a Moldovan woman from an Albanian gang in a deal agreed in a pod on the London Eye. They denied forcing her into the sex trade but were rejected for bail because false documents had been found at their home. A young Londoner who had stuffed £200 worth of Marks & Spencer lingerie down his trousers was remanded for reports.  At last, the list-caller, Angela Georgiou, told the magistrate, Timothy Workman: ‘With sadness, I call your last case ever at Bow Street, sir.’ The honour fell to a 32-year-old Scot from Kirkcaldy, who had breached an Asbo (Anti-Social Behaviour Order) – he was found in possession of a bottle of red wine and some lighter fluid – and little knew that the hand of history had landed on his collar the previous night in the Covent Garden piazza. It was his fifth visit to the court and he pleaded guilty. ‘The closure of the court, with its great legal heritage and its history,’ Workman said, ‘is of great sadness to me and to all who are associated with this court.’ He was the 33rd chief magistrate, a role once held by Henry Fielding, who presided over the first detective force, the Bow Street Runners."  All that was happening the same year, 2006, as we moved to France.

I read a lot, and am frequently caught up in discussions about tablet (Kindle or iPad - I have both) vs the traditional book.  Lots of things jostle for importance in my mind.  Librarians like me were constantly surrounded by books, arranging them on shelves and keeping them by the bed or armchair.  I love our shelves of books, and know most of them pretty well.  So although the electronic formats are less evocative, but more practical for two reasons.  One is often the sheer size and weight: reading in bed holding a large heavy hardback is a struggle, but my Kindle has one other advantage I've come to appreciate reading in French - I can click on a word I don't know and find its translation instantly.  And since we are now of an age when we have to start thinking of downsizing, getting rid of many metres of shelves may in future be a necessity though not yet!  The seemingly limitless capacity of a little Kindle to hold a library of books, and its ability to search for characters whose identity has been forgotten, are great assets.  And, living in France, our regular periodicals like The Week and The London Review of Books arrive right on time.  Mary nevertheless prefers to wait for the paper versions a few weeks later!

We are constantly reminded of litter and waste.  As a dog-walker I  see the endless trail of muck and litter (and of course picking up our own) on the daily trips round the neighbourhood; and of course we have to keep track of regular council rubbish collections, feeling pleased that our own local refuse service has widened out its plastics recycling service, but at the same time wary that we don't really know what happens to the sorted plastics, tin cans and card we throw away.  There is a relatively new, large incinerator in the next village, not at all liked by people living there but clearly it has to be kept busy so one always wonders if some of our waste is not recycled but burnt.  All the same our local authority makes big efforts to encourage us to recycle and has special collections for metal and large electronic items etc - our old computer printer has to be disposed of soon.  And also, the local paper is regularly full of stories and pictures of fly tipping in town centres.  Not a simple process to manage either for us or for the authorities.

My next blog will include a tour of the garden, much changed since our arrival 15 years ago.  The photos each time often chart its simple pleasures, and in the glorious weather we've been having it is a constant joy to watch things growing and glowing.

By [email protected] (Jon North)

May brings looser travel restrictions in France.  The 10 km radius we’ve been confined in was lifted this week, although night time curfews still remain, gradually starting later as we head towards midsummer.  This was the second time we’d had to stay within the circle, and after last time I lost the map, so I am keeping it to hand this time in case it pops up again!  As you can see it would not quite take us to Aigues Mortes or Mauguio, but St Christol was within range, and a corner of the Étang d’Or for flamingo-spotting, but no seaside – our nearest at La Grande Motte is at least 12 km away.

bye bye to one restriction
My life has been disturbed recently by Elvire the dog’s problem of peeing during the night.  It’s easy enough to deal with since we have tiled floors, but it has meant getting up early.  But she seems in good spirits all the same, and our vet has now diagnosed a hyperparathyroid problem which will need surgery – a delicate operation but one which will have to be done, so we are in touch with a surgeon.  Here’s the explanation if you’re interested.  We spend quite' a lot on vets' fees and treatments, but we feel very pleased to have two happy lively dogs, good companions who give us both cause to take a little more exercise than we otherwise might.  The tortoise at least (sadly only one who appears twice below - they are I keep reminding myself they are wild animals and the other escaped more than  a year ago)

As France approaches regional elections, and Presdiential elections are on the horizon, the perennial question 'will the far right make significant gains?' resurfaces.  President Macron is not popular everywhere and tries sometimes to face both ways.  Since France is celebrating the centenary of another controversial leader, Napoleon, this cartoon caught my eye.

Consensual commemoration of Napoleon - the rift between left and right is so out of fashion

Finally a couple of quotes from The Week:  'America innovates, China duplicates, and Europe regulates' (quoted in The Daily Telegraph); and 'At every party there are two kinds of people – those who want to go home and those who don’t. The trouble is, they are usually married to each other (Ann Landers, quoted in the San Francisco Chronicle).  

Despite gradual easing and more vaccinations, there's not much chance of parties to escape from just now...

Animals, an Amelanchier and the wider world

By [email protected] (Jon North)

This weekend the French Covid restrictions will start, cautiously, to be eased - from Monday the 10 km radius for our journeys within France will be lifted though the evening curfew will only be lifted bit by bit.  That affects me taking the dogs for their late evening walk (for which I must carry an attestation until mid May - our shopping lists are written on old attestations for the moment!)  

Our lives have not varied much, but this week we bought a new garden tree/shrub/bush from a newly-discovered nursery not far from here.  We wondered if Amelanchiers, which we had in English gardens, would survive our hotter, drier climate, but were reassured by the man in charge of our excellent local arboretum and by the nurseryman., so our new little bush is proudly in the middle of the lawn near the tortoise enclosure and, lo, just as Mary was planting it the tortoise emerged muddily from her hibernation!  A week or so later than last year, but it is really good to have her back in view.

article about children reading more in lockdown in the Guardian offered me a moment to take notice of the everyday, and to be thankful that my mum encouraged me to read and use the public library. Reading is invaluable, and makes little noise in a time when many media are incredibly noisy. But laughing out loud was, I recall, still risky in my childhood: reading and enjoying Just William books under the bedclothes caused me occasional parental censure, especially when I disturbed the Quaker study groups downstairs! I try to ignore extraneous noise, but it drives me bonkers when someone's Facebook post unexpectedly bursts into speech or music - better to invite sounds in rather than have them blast out without warning (occasionally even waking a sleeping partner at night).

As we continue to mourn the lost opportunity to travel to Armenia again (it would have been a year ago) we take an interest in the terrible difficulties of the people in that tiny country for the sake of our friends there, and one of many good things coming out of the new USA last week was the announcement that President Biden formally recognises the  Armenian genocide.  It is incredible that (kowtowing to Turkey) they have been so slow to acknowledge such a blatant historic human rights abuse.

And among the fairly petty concerns people in the UK and Europe worry about, the appalling situation in countries further away, hooked into hard-line political agendas and too poor to afford the vaccines and treatments, even adequate health services, to cope with a flood of infections.  The people of India are in a profound crisis, somehow in scale with the terrible disasters and episodes of violence in the country’s history. European countries, and the US, have dithered and made false steps - I think we have to remember how fortunate we are. But political dogma and rigidity will always cap ineptitude.

Not my photo, but a beautiful shot of a vine near Pic Saint Loup which survived the recent frosts
Back to plants.  My normal shots of plants are pretty, but these have a whole different focus, and great importance for the future if you’re a wine lover. Looks rather more demanding than your average day in the garden!

At the St Christol Domaine de Coste Moynier

I have good memories of Glendurgan in Cornwall so am delighted to see it at the head of this article, as I am to celebrate spring flowers and blossom everywhere. But this campaign seems to have created a new kind of tree, a ‘blossom tree’. Not sure what that is - most blossom is a transient pleasure on the way to some fruit or other, or in cases like our beautiful grenadier, not due to flower for a month or more yet, categorised as flowering or ornamental trees or shrubs. Ah well, living language...

Podcasts are now a frequent source of information and entertainment for us, and one from the Guardian caught my attention last week - a real-life clown.  Have a listen.  Police spies seem to have been on the radar for ever. I read the book Undercover at least 10 years ago, and it seems to be trickling throughthe judicial process just now.

It turns out a lot of my preoccupations are UK-related - somehow they are more spicy than the French ones I can disentangle!  May day, the fête du travail, is tomorrow, and it has just been announced that the traditional lilies of the valley will be able to be sold, though there are Covid restrictions on roadside sales and it turns out that some of the flowers on sale beside the road are stolen in the first place!

More on words and language

By [email protected] (Jon North)

Our house with its tall conifer - a landmark for those lost around the neighbourhood!

I love maps, but not always on the internet - atlases and other printed maps often seem more satisfactory although less searchable.   You can see from these two maps from the Petit Larousse (too heavy to hold for long, so useful transferred to a screen) that Lunel is right at the eastern edge of the Hérault département.  So many places in the Hérault are actually a long way from us, and many of our contacts and meeting places when we can travel are in the Gard which wraps itself round the right hand side of the Hérault.  Most départements in France are named for rivers - the Gard is unusual in modifying the river name (Gardon), but in any case many rivers spend a lot of their lengths in other départements.  The river closest to us is the Vidourle which is notorious for its floods, every 25 years or so, which forms the eastern border of the Hérault with the Gard.

A scene in our garden some years ago for a lunch after our Tuesday French conversation group. The garden has not changed much but even before the pandemic the personnel has, several of these people have departed from our group in various ways, and there's little chance of gatherings like this again in the near future.

Our life here revolves round the non-stop process of learning French.  After nearly 15 years we've made a lot of progress, but it is still a minefield and equally a source of constant fascination.  There are two dimensions - one, the many-faceted business of written French (easier because you can usually go away and look it up) and spoken French which can vary both in speed (you discover that English people too seem to rattle away too fast) and regional accent, to say nothing of the difference between slang, jargon, technical vocabulary and formal statements.  We have found the Government press conferences models of clarity (even when the news is unwelcome) as are magazine tv shows like the excellent travel magazine programme  Des racines et des ailes, but catching chatter among friends is often difficult, as if speaking and hearing on the phone, especially when you don't know in advance the topic of the call.  Sadly we have all too little friendly chatter at the moment.

The first poppies a week or two ago

The issue of sexist language is very topical here now.  France is some way behind the English-speaking world in this, partly because of the additional complexity that all nouns have gender, and historically because of the Académie française which has for centuries laid down the law about what is correct and incorrect.  Now inclusive names of the 'man includes woman' kind are much less common - names of professions (professeuse as well as professeur, for instance) have increasingly been feminised since the 1980s and mor recently there have been official circulars defining how inclusive pronouns like 'he or she' should be written - I have always tended to use / , but one hotly debated method here is the use of the point médian, a kind of decimal point when for instance masculine and feminine endings are indicated, for example professeu·r·se.  As with much to do with language in French, this gets rather complicated (just typing the thing is complicated enough!)

Such cultural debates are often long and heated - a supplement in Midi Libre was headed to "cette écriture qui divise tant" and referred to a veritable (ardente) battle of Hernani - what on earth is that?  Well, it's a reference to a play Hernani by Victor Hugo (the name is from a Spanish town).  It became notorious for the demonstrations at its first performance between the romantics (Hugo had Berlioz and Théophile Gautier, poet of Les nuits d'été' among his champions) and classicists who saw the play as a direct attack on their values.  Verdi also wrote an opera, Ernani, apparently.  So passions were roused then, and people today recognise the reference nowadays well enough for it to be used in a local newspaper.

Clearly understanding culture, popular or not, is a complicated process.  Meanwhile, we listen to frequen official press conferences on Covid as much to hear the very clear language ministers and officials use as to keep up with the latest information.  Restrictions in movement are to be eased, and in any case we have now both been vaccinated, but our lives are quiet and the chance of travelling to England is very small just now, so sadly our family and friends there remain at the end of phones and video calls.  This week the physiotherapist, Emma, who has helped me through over 3 months' healing of broken arm is leaving for her next placement, working with children in the hills somewhere.  I'll miss her, but of course the team practice she works in will provide me with a replacement.  And April continues to bring mostly beautiful weather.

Roses starting to show in the front yard

Solemn news, and, What's in a name?

By [email protected] (Jon North)

National figureheads aged all but 100 do not die often, so the passing of Prince Philip deserves more than a mention.  ‘Figurehead’ is an apt description for this naval man whose function was mainly ornamental. 

These days my day starts early, since Elvire the dog needs letting out.  This Saturday the usual selection on Radio 3 (and I’m guessing all over the BBC) was replaced with nearly continuous music which is not disagreeable.  It leaves space for reflection - I’m not a royalist but I think the mood was right at a time of national mourning following the death of Prince Philip at a venerable age.  For my friends who already think there is too much fuss about posh people in the media, I find the response of wall-to-wall music ideal.  And despite the increasing rarity of chimney sweeps, I recall Shakespeare’s verse

Thou thy worldly task hast done,
Home art gone, and ta’en thy wages:
Golden lads and girls all must,
As chimney-sweepers, come to dust.



Here are the 20 top names (boys and girls, 40 in all) registered in Montpellier over the last 20 years.

Local expats (people like me, but I don't share their views!)) commented that the names were ‘not very French’.  But two things strike me – the people commenting are probably much older than the people (many being parents of the new babies) who chose the names, even if not as old as me.  One comment expressed surprise  that Catholic names do not figure more prominently – but France is a secular state and prides itself on its republican values, even though saints’ days still figure on all published diaries and calendars, and many public holidays are still church hangovers. 

 So yes, we live in a very contradictory country!  Second, saying things are not very French is a bit odd, since they are by definition French choices. I find the absence of more north African names a little surprising given the significant proportion of families of north African  origin in the area.

Among the podcasts we’ve listened to recently is Tim Harford's Cautionary Tales.  These are really good at adding a dose of reality and realism to the mad hype of modern life.  As well as reality checks on the pandemic, his subjects range from the Charge of the Light Brigade to Harold Shipman, and art forgery, and much, much more.

Our weekend has also been enlivened by watching two Rugby League Challenge Cup matches - the only French representatives, Catalan Dragons, gave a very spirited performance to win their tie against Wakefield Trinity.  Rugby here is almost all Union rather than the rugby à treize we were watching, but the Catalan team is more or less local to us, and we've always enjoyed watching this form of the game.

Meanwhile the news here has also been full of  grim news of frozen vines and fruit trees - lots of growers have lost their crop this year thanks to at least one night of temperatures as low as -8°C.  Not uncommon further north but pretty unusual here, and thus less likely to be insured.

Vines bursting into leaf near Luc's garden in Aigues Mortes, befiore the very cold weather  

More hopeful signs of spring with our latest dwarf irises

Spanish winter

By Jon North ([email protected])


My encounters with Spanish wines go back to my teens, when my father returned from holiday in Spain (with my mum of course, but I’m not sure if Pam and/or Tom went too) lugging on the train 12 bottles of Rioja.  We have carried many bottles in our car over the years, but on the train with all the suitcases too…  Only my Dad!

More recently it all started in the autumn when Mary sighed that she missed Fino sherry, and we both recalled our love of PX (short for Pedro Ximenes, a grape variety used for sweet fortified wines in southern Spain, and so used in blends with drier wines to make the ubiquitous 'cream sherries' well-known in the UK).  We first encountered 100% PX 20 years ago at Rick Stein’s Padstow restaurant when the dessert was simply vanilla ice cream with a schooner of PX poured over.  

So I hunted idly to try and satisfy Mary’s craving and came across the Barcelona wine merchant Decántalo.  They specialise in Spanish wines but a quick scan shows they have plenty from other countries too – it seems to me to be as reliable and well-organised as the UK Wine Society with the advantage for us that it is just down the road, not in what we have now sadly to think of as a foreign country!

On top of this, this wine merchant packs things in super-safe cartons – we've had one or two disasters with breakages in the past, but bottles really have a hard time getting broken in these deliveries.  We began before Christmas with Tio Pepe Palomino Fino and Gonzalez Byass Nectar PX, plus a couple of  unfortified wines.  One of the first of these we tried was Protos Clarete, a light red or dark pink from Ribera de Duero, an area now quite well-known, top centre/north Spain a bit SW of Rioja.  This reminds me of the light red clairet which is produced in Bordeaux and whose name of course links to the red clarets from there we all know and love.  But the dark pink, or light red, style is distinct there as it turns out to be in Spain.

We also tried 3 Muga Riojas (red, white and rosé), all of them very good – a friend who knows this whole field well says white Rioja is hard to find, and remembering Dad’s samples I was really interested in this – Oz Clarke says white Rioja is a bit of an afterthought, but we have really liked this Muga, so still have some bottles to enjoy!   We then spotted a Christmas offer of 18 different wines, all red,

·         6 from Ribera del Duero (see above – we have already enjoyed these, wines and makers included Juegabolos, Malleolus, Corimbo I, Pago de los Capellanes Reserva, Pícaro del Águila and Astrales);

·         6  from Priorat(wines and makers include Ferrer Bobet Vinyes Velles, Salanques, Laurel, Mas d'en Compte Negre, Planetes de Nin, and Coma Vella) – we are halfway through these and have been so impressed by the  Ferrer Bobet that we have spent a small fortune on some of these bottles to lay down for the next few years;

·         and 6 other reds, 3 from Montsant: (Fredi Torres Lectores Vini Montsant; Sindicat La Figuera; and Orto - we shared the  Figuera wine with friends yesterday and were very impressed), another Rioja: (Diez-Caballero Crianza 2018) and two from further south (Jumilla - SE Spain N of Murcia: Casa Castillo; and from Valencia: Parotet Vermell)

In the huge country of Spain and its many contrasting wine areas, I was surprised to find that two of these 3 appellations are tucked into a tiny hilly area SW of Barcelona: 

Priorat,  which according to Hugh Johnson's pocket guide produces some of Spains finest wines. It's named after a former monastery tucked under craggy cliffs. The key is the slate soil – known as llicorella. 

Montsant, tucked in around Priorat 

The thing to look forward to is a trip to these areas of Spain, relatively close to us although the travel complexities of travel with Covid, national restrictions and of course Brexit remain to be discovered! 


Springing forward

By [email protected] (Jon North)

This week's blog is mostly about health.  The good news this week is that my doctor has fixed Covid vaccinations for both of us today, which is a relief.  Of course, the rapidity with which vaccines has been developed and made safely available is pretty extraordinary.  It's interesting how many of the fears about vaccine denial have faded, though there is still negative babble at the fringes - I think most people see it as the only possible route back to 'normality', though there are hundreds of questions about what that should be or how things may change.

But the unholy tangle of new strains being discovered, problems about bulk manufacture and distribution of vaccines, political mud-slinging and bureaucratic delays, justifiable or unjustifiable caution about safety and side-effects, all of these created an unsettled atmosphere in ordinary people who worry about jobs as well as getting ill, with more time than usual to ponder things.  So other quite trivial things - for example the jamming of the automatic ignition on our gas hob, issuing a stream of crackling sparks for 24 hours - add yet more tension, and it took longer than it should have done to discover the off switch!

Hopeful buds on a drought-hit lemon tree, recovering this spring

I've written before about my physiotherapy, mostly a very positive experience this time (previous sessions of treatment for knee and sciatica were far less useful).  But I keep on realising how much better things will be as and when I can take responsibility for my own exercise and recovery.  When I was in a clinic after my knee operation I spent one hour out of 24 actually being treated; now I have a good half hour's intensive massage, and 20 minutes a session wrapped in a kind of cold blanket where icy water is pumped through.  And above all I have exercise machines at home which I can use when it suits me.  I've kept up the static bike throughout, and now I'm about to experiment gently with the cross-trainer whose arm movements seem to fit into what is allowed, applied gently!

Dogs meanwhile provide a structure and timetable to our days.  I've been able to resume early morning and late evening short walks, which takes pressure off Mary.  They have some digestive and bladder problems which need vet checkups.  But although these vet sessions are expensive, they are also reassuring, and the dogs are OK, and continue to provide us with enjoyment.

Marching on, and on

By [email protected] (Jon North)


Tom with me and sister Pam

13 March would have been my brother Tom's 70th birthday. He and his family are very much in our minds - departed far too soon. 

Meanwhile 12 March was another milestone in progressing our established status as French residents. That morning Mary and I surrendered our temporary EU cartes de séjour at the préfecture; our new cards as permanent foreign residents should arrive by post in a few weeks. Despite gripes elsewhere I have always respected French bureaucracy - if you follow the instructions you get the right results. The woman who dealt with our dossiers was very friendly and the whole rendez-vous more or less on time. We even had time, in a strangely quiet Montpellier, to buy some excellent kitchen gear before heading home to the dogs by midday. A good morning’s work.

There was some other welcome news for we Britons living abroad this week, or at least a promising development.  Buried in the details of the UK budget was news that the government plans to scrap the rule that bars those who have lived abroad for more than 15 years (as we shall soon have done) from voting in UK elections "the government is providing an additional £2.5m to remove the limit preventing British citizens who live overseas from voting after 15 years.”  So we shall be able to continue to vote in the West Derbyshire constituency where we lived before leaving the country.  

One story in the local paper caught my eye - of the revival of a traditional industry making fabric in Nîmes.  The blue denim, named ‘de Nîmes’ by the Americans who popularised it for jeans, is now going to be manufactured by a family firm in this area.  Among other local news, another story this month of wartime good deeds - A Jewish man has bequeathed a sizeable chunk of his fortune to the French village whose residents hid him and his family during the Second World War, despite the risk to their own lives. Eric Schwam, who died aged 90 on Christmas Day, is believed to have left s2m to the remote mountain village of Le Chambon-sur-Lignon, which gave shelter to 3,500 Jews during the war. Schwam and his family arrived in 1943 and stayed until 1950. In his will, he said the gift was an expression of his gratitude, and suggested it be spent on youth services.  We have friends whose family shared in the efforts to rescue Jewish people in this part of France during the war -  this strikes many chords with me.

This cartoon is for our daughter-in-law and many other friends involved in theatre and music. The Paris Odéon-Théâtre de l'Europe has been occupied by protestors, with Union support demanding reopening of theatres and extension of government support to the so-called intermittents who would normally be working there. The Minister Roselyne Bachelot has intervened but has not apparently been able to reassure all protestors. 

Sport has been a frequent interest for us as always.  I'm still reeling a little from the turbulent reversal of fortunes in the final stage of the Paris-Nice cycle race. Having led for most of the week, Primoz Roglič ended up 16th overall with ripped clothing and having dislocated a shoulder, and the winner in the end was Max Schachmann who also won last year. Riveting watching all the same.  Meanwhile the 6-nations rugby has caught fire despite the absence of spectators with a fizzing match between France and England and an absorbing on between Scotland and Ireland.  Meanwhile the woes of 'my' football team Liverpool continue, but it is after all only sport.  Another picture to end with.

The McMansion Hell Yearbook: 1980

(back of a quirky literary novel voice): Sometimes, things are not what they seem. An architecture critic disappears for three months to follow bike racing around Europe, rife with questions of becoming and desire. A real estate agent uploads a listing to an aggregator, knowing that it will be a difficult sell but thinking not much of it, for, like Tolstoy’s unhappy families, all houses are difficult to sell in their own way. A house is built in 1980 in Staten Island and would have thrived as an anonymous bastion of tastelessness had the internet not been invented. But the internet had been invented. All of these things are brought together here, through truly unlikely circumstances.

Let’s not bother with the formalities this time.

None of you will buy this house.

Sitting Room

Does anything here make sense? The periwinkle sofa, the twinkling of bronze glass, a truly transitional material, a mall exiting stagflation and entering the sultry trap of Reaganite libertarianism that would leave it empty twenty-five years later. The sense that one is always changing levels, trapped in a landing of some sort, never quite arrived on stable footing. But that’s just the style, one assumes. One foot in the seventies, with all their strife, one foot in the beginning of what felt like the end of history. One’s ass on the iridescent pleather sofa, waiting for the centuries to change.

Sitting Room II

My suspicion is that there are no pictures of the mirrored mystery foyer because the photographer’s identity would be henceforth revealed, and the point of all real estate photography is for the viewer to imagine themselves as the only person in a given space.

Dining Room

The shinier things are, the richer one is, obviously.


This serious sociological research also happens to coincide with the Giro d'Italia, one hopes.


(crediting @cocainedecor on twitter for their term. but also, where can i get some chevron mirrors, asking for a friend.)

Master Bedroom

just asking questions

Bedroom 2

Ostensibly bad opinion that I will nevertheless defend: the corner bed slaps, let’s bring it back.


(Staten Island accent): Hey, I’m workshoppin’ some metaphors here!

Alright, we’ve entertained this monstrosity enough - time to wrap things up.

Rear Exterior

You know, McMansion Hell has been around for five years now, and has coined many terms - an art, ahoy matey, lawyer foyer, brass n’ glass, pringles can of shame - but I have to say, I hope fireplace nipples also sticks.

Anyway, that’s all for 1980 - join us next month for 1981.

If you like McMansion Hell, support it on Patreon!

short lease in a slick machine: a personal essay about apartments

Hi Everyone, you may have wondered where I’ve been for the last few months. The truth is, I, like most people must at some point in their lives, needed to take a little break and figure some things out, needed to go on some long personal journeys, needed to meet some heroes, needed to just not do this website for a short amount of time, but don’t worry, I’m back now, and I’m bringing the feels on the way in.

Before I present this essay, I would like to offer my deepest thanks to the people who kept supporting me on Patreon during this soul searching. I owe you everything.

I’m moving again. I’ve moved every single year since I’d left my parents’ house at the age of eighteen, with the exception of the apartment I had on the second story of a Queen Anne on S. Mendenhall Street in Greensboro, in which I stayed in for two years. The rest of my dwellings have been painfully temporary, with life inevitably coming around to its annual migratory upheaval. There have been many reasons why, of course, quotidian reasons that always feel devastating at the time – jobs, school, pestilence, crazy roommates, despicable slumlords, partners to be moved closer to, relocating just to get away from where one has been before. I could rank every apartment on a scale of worst to best, from most to least livable, but none of them were permanent.

above: the only apartment I ever lived in for more than a year, a sacred place.

I wanted to write about the apartment I’m moving away from in Chicago even though perhaps it’s not prudent to do so – it’s never prudent to be personal on the internet. Don’t worry, though, I won’t include anything incriminating that could be construed as defamation or whatever. You can just feel angry on my behalf, which is really, truly in the spirit of McMansion Hell. And this is, well, apartment hell. The apartment I’ve lived in this past year quite frankly and very succinctly encompasses everything I kind of hate about architecture, about design, about the ways people in the profession are expected to live their lives for the benefit and the consumption of others.

first impressions

When I first saw the apartment, it was the nicest apartment I’d ever been in, the finest I’d hitherto walked the halls of in my rubber Birkenstocks. It was big and full of light, with lovely maple floors, the kind where, at the right time of day, you could sometimes see the tiger pattern emerge in flecks and ribs like those on the backs of violins. When the landlord, an architect, showed it to us, he had his stuff in there still. A Bertoia chair that was probably real. Very carefully selected items from Design Within Reach alongside enough pieces from other places to make the whole getup seem more authentic. Sparse hangings on the walls, each big and well-framed. Single potted plants. A well-oiled cutting board.

There were European bath and kitchen fixtures and recessed lights that dimmed at the press of a button, which meant we could get rid of all of our floor lamps. In the kitchen, tall, elegant white cabinets above a slab of marble, dubbed, reverently at the time, a living material. Blinds on rollers meant no need for hanging curtains. A soaking tub and a Duravit toilet, you know, the floating kind cultured people had. Europeans. The rent was at the top of our budget but still doable. I signed the lease fast, with unbelievable giddy excitement. Finally, a nice place to live after years and years and years in what could only be deemed as shitholes. Shitholes and the nice midcentury apartment building I lived in in DC, but that was a studio and DC was a place I wanted to get so immensely far from that we ended up in Chicago, the only city in America I ever really wanted to live in.

cracks in the facade, so to speak

As soon as we moved in, an unsettled feeling crept in. I can place it now as the sense that this apartment was too nice for people like us – people with particle board furniture and student loan debt. That it wasn’t really ours, we were just borrowing it before someone worthier came. Subconsciously, we knew this. We never hung anything on the walls save for the Mondaine clock my husband bought at the MoMA Design Store and the Giro d’Italia jersey signed by Tom Dumoulin, which I’d had framed. The walls were a blinding white. Putting tacks in them felt like an unlawful penetration. Our landlord fussed over the stuff we had on the back porch. One time he criticized where my husband had situated the soap on the kitchen counter, the living material which, in reality, is just a fancy term for “stains easily.”

All of a sudden, we were living under a microscope.

We weren’t using the apartment the right way; namely, we didn’t decorate or live like an architecture critic and a mathematician theoretically should. Our apartment wasn’t photogenic. There were too many bikes in the living room. We still had a garbage $300 Wayfair sofa that felt like sitting on cardboard. There was clutter. This beautiful apartment wasn’t meant for our kind of ordinary and this was made known several times in subtle and rather degrading ways, after which our lease was not renewed, to the relief of all parties involved. Even if it meant moving again.

The longer I lived in the apartment, the more I hated it, the more I realized that I had been fooled by nice finishes and proximity to transit into thinking it was a good apartment. As soon as we’d got in there, things started to, well, not work. European fixtures aren’t well-liked by American plumbers. The dimmable lights would sputter and spit little blinking LEDs for reasons totally unknown and we’d have to pull a tab to reset them. Everything was finicky and delicate. The shower head, the kitchen sink that fell in two times somehow (which we had been accused of being rough with, an absurd thought – it’s a kitchen sink!), the bedroom doors that didn’t close right, the bathroom door that would trap you inside if it shut during a hot shower. All of the niceness, the glitzy brand names, the living materials were not meant for everyday use, even by gentle individuals like ourselves. They were made solely for looking at, as though that were the point of all habitation.

Suddenly, we were in a prison of design. This was a place for performing living, and we, as normal people, simply wanted to live – wanted to leave clothes in front of the washer as we pleased, wanted to bake cakes that got flour everywhere, wanted to just collapse somewhere and go to sleep, wanted to have a private life not dominated by the curation and fussiness and pressures of taste that govern careers like mine. Our house was always just for our consumption, not that of others. I spend most of my life in the worlds of design and architecture, and to be honest, you wouldn’t know it aside from all the heavy books and the tapered legged coffee table. I never had it in me to turn my house into a museum of my own clever delectations, a proof of concept of my skills as a critic. I just wanted to dwell naively. Off Instagram.

But the worst part of the apartment was that it was designed by someone who didn’t know how to live, couldn’t think of anyone’s world other than the sparse one of the architect who owned nothing save for color-coordinated books and limited edition lithographs. It had all the functions of living, technically speaking, but the way in which they were allocated and arranged made no sense. There were no closets in any of the rooms, just open storage, which only works for people who don’t actually have things. The tub wasn’t caulked to the wall so that it would appear to float, a nice aesthetic effect which made taking showers annoying and perhaps bad for the walls.

Above all, I hated the kitchen the most. The kitchen was basically ten feet of counter space, with giant cabinets extending to the ceiling, far beyond what any normal person could reach without a stepladder, the upper shelves of which being where things went to be forgotten. A sink punctuated the center of the marble countertop – and marble is a terrible material for a countertop. It stains and wears with water. It shows all mess mercilessly. There was a stove and a fridge just, like, in the kitchen attached to nothing. The gas stove had no overhead ventilation and every time we used it we had to open the door so the smoke alarm wouldn’t go off. It was a kitchen designed by people who never cooked: too small, inefficient, laid out in the way it was, like so many apartment kitchens, so that it shared services with the same wall as the bathroom. We couldn’t put anything in the finicky sink to soak so the counter was always crowded with dishes. We had no dishwasher because that would mean ceding the only bottom cabinet that was truly usable.

It angered me, really, as an architecture critic, that this apartment, which had so very much been made to be ogled and looked at and oohed and ahhed over by people of taste was absolutely, for a lack of a better word, bullshit. That it was beautiful but unlivable, like some kind of joke made only for people like me to laugh at. I love design, obviously, but I hate the pressure to have to perform taste in the most intimate of one’s settings and this was the epitome of that, the untouchableness of it, the smug superiority of its flavorless emptiness. I’m not a curator of other people’s gazes when I’m in my pajamas or sweating it out on the trainer. I’m simply Kate Wagner, living with a husband and a dog, like a lot of twenty-seven year old white girls in cities. By the end of the lease, I just wanted to move somewhere where I’d feel at home, whatever that meant. I never had the type A personality needed for pristine white walls. I hated how the recessed lights made all our stuff look cheap, like a museum of stunted adulthood.

Our new apartment has a two-year lease, which is about as much stability people like us could ever hope for or afford. It’s the first floor of a worker’s cottage dominated by a palladian window on the second story that would be pretentious were it not so earnest. The house itself is a hodgepodge of the vernacular, which is what I deserve, as its chronicler. The interior walls are painted lively colors – a soft blue, a slate purple, a taupe, a mint green. It’s gritty enough to be cool and old enough to be livable. There are closets. The bathroom is covered in chiclet glass tile that’s different shades of blue, which I find endearing. But what I love most of all is the kitchen.

All my life, I’d been in search of an apartment with a decent kitchen, and I’ve always wondered why apartment kitchens suck so bad save for the obvious answer (landlords are cheap.) Like I said earlier, the desire to route services (plumbing, electricity) in the most efficient way possible governs most things, though this is more true of renovations or new builds than the adaptation of single family homes into multi-family dwellings. In the case of the latter, the second floor apartments are always the worst off, in fact, almost all apartments are worse off than the one that houses the actual original full-sized kitchen to begin with.

Adapting a space that was meant for sleeping into one where food could be cooked often required some inventiveness with regards to fire safety and ventilation and this usually took the path of least resistance, hence why most kitchens are positioned to the rear of the house, especially if there is outdoor access. (Plumbing in older houses also tends to be positioned on interior walls to avoid pipes freezing in the winter.) In Chicago, most layouts of familiar single-family vernacular housing styles are similar to one another on the ground floor, but the apartments on the second floor are always quite varied, especially with regard to where the kitchen is placed. Often it’s done, again, in a way that allows existing services to be used or for new ones to be built that are on the same wall as another unit. Adding new plumbing where it wasn’t before is expensive and a pain.

However, service routing aside, most apartment kitchens are only ever satisfactory – kitchens for people who ate nothing but takeout or miniature versions of the real thing as though apartment living were just an audition for owning a house, something that’s just no longer true in this economy. This one – with its vintage 50s aluminum cabinetry and its enameled countertops with glitter infused in them like some kind of demure bowling ball and its full-sized appliances and dishwasher, and mint green penny tile, its wonderful quirkiness and its ample cabinet space beneath the counters – is functional. It works like a kitchen should, towards a domestic life engineered by modernism and scientific management with a dash of feminism to be less arduous. This is nothing short of a miracle to me. When I think about it, I get emotional. I have been searching for so long for any kind of semblance of a place tailored in any way towards my needs, towards my desires, which is to have enough space to help rather than hinder in the preparation of meals. Meals we now enjoy as a very small family. The kitchen was never really important to me until I had someone to share it with, as insipid and mawkish and introduction-to-a-gluten-free-recipe as that sounds. I’m no longer living for one, but for two, and I didn’t realize how much that changed living.

I didn’t realize how much autonomy meant until I lived in a place where I felt I had none.

Our new landlords, a school-teacher and private investigator (what a combo) are there right now cleaning the house, fixing the little nicks left by the previous tenants, pulling out their picture hanging apparatuses, which, they assure us, we can leave too. We can put stuff up on the walls, the very thought! They’ve already stickered our names on the mailboxes, have installed a doorbell, which strikes me as a very post-COVID gesture. They hope we will stay there a long time, and so do we. There’s a yard for the dog to play in with garden beds that house burgeoning bell peppers. Our friends are allowed to come over, which they weren’t before — well, not officially, but it felt like it. There are sounds in the house, of those who dwell above and below, the sounds of life. There’s a window I wish I was sitting by writing, and soon, I will be.

So many of us ask the simple question, what is home? What should it be? And the only real, genuine answer I have to give after ten-odd moves is that home is the only place in the world where one can be truly unselfconscious. Even if that means having particleboard furniture and a bunch of bicycles.

That’s my business, not yours.

If you liked this post, check out the McMansion Hell Patreon, or, if so inclined, drop a tip in the tip jar.

r/McMansionHell - Hello r/McMansionHell, I'm Kate Wagner, creator of McMansion Hell and architecture critic at The New Republic. AMA!

r/McMansionHell - Hello r/McMansionHell, I'm Kate Wagner, creator of McMansion Hell and architecture critic at The New Republic. AMA!:

Howdy, Folks! If you’re not busy right now, I’m doing an AMA on the McMansion Hell Subreddit! Stop by and ask me some questions!

The McMansion Hell Yearbook: 1979

Howdy, folks! I hope all of my fellow Midwesterners are enjoying this year’s false spring. Seventy-degree days notwithstanding, the snow will indeed be back, and, as such, I have prepared for you a house to enjoy (?) alongside a miserably late-in-the-year hot cocoa. 

Now, this house isn’t as oppressively horrible as the last one, however, the point of the Yearbook is to show off how houses evolved overtime, and also to celebrate some of the kookier time capsules left out there. Our current house falls into the latter category, and to be honest, I find it weirdly endearing. 

Located just outside of Detroit, this 5 bed, 4.5 bath house tops out at over 10,000 square feet. Yes, you read that right. 10,000. You’ll see why later. Anyways, if you want to purchase said house, it can be all yours for just under $1,000,000. A steal!

??? Foyer

In America we don’t have barons, only robber barons, so I’m going to assume whoever built this house did so on the backs of thousands of exploited 19th century child laborers or whatever. Bad stuff. 

??? Room

Unsure of the purpose of this room, genuinely, because all other rooms are accounted for. This one’s just empty. It’s just existing. Vibing, as one might say.

TV (???) Room

Don’t mind me, I’m just getting out my birding binoculars in order to watch Seinfeld reruns.


Considering the history of the Midwest and the fact that Sears and co. cut down all our old growth forests in order to do cheap furniture and balloon framing, this whole wood paneling bit is really part of a much larger historical milieu.  

Dining Room

The American Bicentennial lurks in the background of all of these houses, its legacy permanently ingrained in too-dark rooms across the nation. 

Main Bedroom

Shivering at the thought of my feet touching cold tile floor every morning. That’ll wake you up. 

Other Bedroom

You know, the grandness of the chandelier has diminishing returns if you put one in every single room. Then it becomes just another light fixture. 

Random Bathroom

BROWN TUB BROWN TUB BROWN TUB (the rarest of all mid-century tubs)

Pleasure Grotto

Ok now this is why I chose this house. It also explains why this house is 10,000 square feet - at least half of that is just this pool alone. The funniest bit is, I can’t for the life of me tell WHERE this pool is by looking at the exterior of the house. In fact, I’m not sure how they managed to fit so much house in that small of an envelope, but at this point, it’s so weird I’m inclined not to ask further questions. Some things in the universe are not meant to be known to us. 

Rear Exterior


Anyways, I’ll let that haunt you for a little while. 

In the mean time, I’d like to take this space at the end of the post to announce that I’ve started a little side project devoted to my other love in life, professional cycling. It’s a newsletter called derailleur that aims to tell the stories of contemporary professional cycling in an unconventional, narrative-driven way. If you’re into such things, feel free to check it out: 


If you like this post, and want to see more like it, consider supporting me on Patreon!

There is a whole new slate of Patreon rewards, including: good house of the month, an exclusive Discord server, monthly livestreams, free merch at certain tiers and more!

Not into recurring donations but still want to show support? Consider the tip jar!

Or, Check out the McMansion Hell Store! Proceeds from the store help protect great buildings from the wrecking ball.

i draw cyclists

i draw cyclists:

hey so in my spare time, when i am not an architecture critic, i like to draw. this has merged with my other love in life (professional cycling) and as such i have dedicated the last six months of my evening hours to drawing cyclists. 

tl;dr i made a tumblr where i’m posting my art if you’re into that kind of thing. this isn’t monetized in any way and i don’t take commissions - it’s just a nice (if strange) hobby i enjoy.

see you all monday with a new house post <3

The McMansion Hell Yearbook: 1978

Howdy Folks! Today’s house comes to us from Iredell County, North Carolina, and trust me, it is quite a doozy - just in time for Valentines Day, too! If you don’t fall in love with it, I don’t know what to tell you. 

This 5300 square foot, 4 bedroom, 4.5 bath house, comes in at $625,000, making it more of a bargain than most McMansions usually are, and while the Tudors never came to America, a place that had not yet been “discovered” by the time the Tudors were in power in England, fear not - for all the repression and stuffiness of 15th century Britain can still be found within these darkened doors. 

Lawyer Foyer

If your house doesn’t constantly give off I AM MARRIED vibes, your spouse might start having indecent thoughts. One must stay vigilant at all times. 

Dining Room

Look, hutches are good storage, okay. Sturdy. We as a generation (millennials) need to get back into knickknacks. Minimalism is dead. Long live kitsch. 

Living Room

Honestly, this house is so dark and repressed it makes high school me look like a libertine. 


“What do you mean ‘dopeness’ isn’t a qualifier for granting a property historical landmark status?” 

Main Bedroom

Love is in the air. Also the air is really, really stale in here right now. 


If your bathroom doesn’t emulate a luxuriant grotto, wyd???

Bedroom 2

please, my floor ducks, they are so cold,,,,


I have got to stop using epic ironically. I already lived through 2008 once. 

That’s it for the interior! Let’s just step outside for a quick breather…

Rear Exterior

Well, I hope you had a good time traipsing through what can only be described as a treasure trove of different matching fabrics. Be sure to stick around for the next part of “Underground” which is coming your way shortly!

If you like this post, and want to see more like it, consider supporting me on Patreon!

There is a whole new slate of Patreon rewards, including: good house of the month, an exclusive Discord server, weekly drawings, monthly livestreams, a reading group, free merch at certain tiers and more!

Not into recurring donations but still want to show support? Consider the tip jar! 

Or, Check out the McMansion Hell Store! Proceeds from the store help protect great buildings from the wrecking ball.

Underground, Part 1

[Author’s Note: A year ago, when waiting for the DC Metro, I came up with an idea for a short story involving two realtors and the infamous Las Vegas Underground House, typed up an outline, and shoved it away in my documents where it sat neglected until this month. The house recently resurfaced on Twitter, and combined with almost a year of quarantine, the story quickly materialized. Though I rarely write fiction, I decided I’d give it a shot as a kind of novelty McMansion Hell post. I’ve peppered the story with photos from the house to break up the walls of text. Hopefully you find it entertaining. I look forward to returning next month with the second installment of this as well as our regularly scheduled McMansion content. Happy New Year!

Warning: there’s lots of swearing in this.]



Back in 1997, Mathieu Rino, the son of two Finnish mechanical engineers who may or may not have worked intimately with the US State Department, changed his name to Jay Renault in order to sell more houses. It worked wonders.

He gets out of the car, shuts the door harder than he should. Renault wrinkles his nose. It’s a miserable Las Vegas afternoon - a sizzling, dry heat pools in ripples above the asphalt. The desert is a place that is full of interesting and diverse forms of life, but Jay’s the kind of American who sees it all as empty square-footage. He frowns at the dirt dusting up his alligator-skin loafers but then remembers that every lot, after all, has potential. Renault wipes the sweat from his leathery face, slicks back his stringy blond hair and adjusts the aviators on the bridge of his nose. The Breitling diving watch crowding his wrist looks especially big in the afternoon glare. He glances at it.

“Shit,” he says. The door on the other side of the car closes, as though in response. 

If Jay Renault is the consummate rich, out-of-touch Gen-Xer trying to sell houses to other rich, out-of-touch Gen-Xers, then Robert Little is his millennial counterpart. Both are very good at their jobs. Robert adjusts his tie in the reflection of the Porsche window, purses his lips. He’s Vegas-showman attractive, with dark hair, a decent tan, and a too-bright smile - the kind of attractive that ruins marriages but makes for an excellent divorcee. Mildly sleazy.

“Help me with these platters, will you?” Renault gestures, popping the trunk. Robert does not want to sweat too much before an open house, but he obliges anyway. They’re both wearing suits. The heat is unbearable. A spread of charcuterie in one hand, Jay double-checks his pockets for the house keys, presses the button that locks his car. 

Both men sigh, and their eyes slowly trail up to the little stucco house sitting smack dab in the center of an enormous lot, a sea of gravel punctuated by a few sickly palms. The house has the distinct appearance of being made of cardboard, ticky-tacky, a show prop. Burnt orange awnings don its narrow windows, which somehow makes it look even more fake. 

“Here we go again,” Jay mutters, fishing the keys out of his pocket. He jiggles them until the splintered plywood door opens with a croak, revealing a dark and drab interior – dusty, even though the cleaners were here yesterday. Robert kicks the door shut with his foot behind him.

 “Christ,” he swears, eyes trailing over the terrible ecru sponge paint adorning the walls. “This shit is so bleak.”

The surface-level house is mostly empty. There’s nothing for them to see or attend to there, and so the men step through a narrow hallway at the end of which is an elevator. They could take the stairs, but don’t want to risk it with the platters. After all, they were quite expensive. Renault elbows the button and the doors part. 

“Let’s just get this over with,” he says as they step inside. The fluorescent lights above them buzz something awful. A cheery metal sign welcomes them to “Tex’s Hideaway.” Beneath it is an eldritch image of a cave, foreboding. Robert’s stomach’s in knots. Ever since the company assigned him to this property, he’s been terrified of it. He tells himself that the house is, in fact, creepy, that it is completely normal for him to be ill at ease. The elevator’s ding is harsh and mechanical. They step out. Jay flips a switch and the basement is flooded with eerie light. 

It’s famous, this house - The Las Vegas Underground House. The two realtors refer to it simply as “the bunker.” Built by an eccentric millionaire at the height of Cold War hysteria, it’s six-thousand square feet of paranoid, aspirational fantasy. The first thing anyone notices is the carpet – too-green, meant to resemble grass, sprawling out lawn-like, bookmarked by fake trees, each a front for a steel beam. Nothing can grow here. It imitates life, unable to sustain it. The leaves of the ficuses seem particularly plastic.

Bistro sets scatter the ‘yard’ (if one can call it that), and there’s plenty of outdoor activities – a parquet dance floor complete with pole and disco ball, a putt putt course, an outdoor grill made to look like it’s nestled in a rock, but in reality better resembles a baked potato. The pool and hot tub, both sculpted in concrete and fiberglass mimicking a natural rock formation, are less Playboy grotto and more Fred Flintstone. It’s a very seventies idea of fun.

Then, of course, there’s the house. That fucking house. 

A house built underground in 1978 was always meant to be a mansard – the mansard roof was a historical inevitability. The only other option was International Style modernism, but the millionaire and his wife were red-blooded anti-Communists. Hence, the mansard. Robert thinks the house looks like a fast-food restaurant. Jay thinks it looks like a lawn and tennis club he once attended as a child where he took badminton lessons from a swarthy Czech man named Jan. It’s drab and squat, made more open by big floor-to-ceiling windows nestled under fresh-looking cedar shingles. There’s no weather down here to shrivel them up.


“Shall we?” Jay drawls. The two make their way into the kitchen and set the platters down on the white tile countertop. Robert leans up against the island, careful of the oversized hood looming over the electric stovetop. He eyes the white cabinets, accented with Barbie pink trim. The matching linoleum floor squeaks under his Italian loafers. 

“I don’t understand why we bother doing this,” Robert complains. “Nobody’s seriously going to buy this shit, and the company’s out a hundred bucks for party platters.”

“It’s the same every time,” Renault agrees. “The only people who show up are Instagram kids and the crazies - you know, the same kind of freaks who’d pay money to see Chernobyl.” 

“Dark tourism, they call it.”

Jay checks his watch again. Being in here makes him nervous.

“Still an hour until open house,” he mutters. “I wish we could get drunk.”

Robert exhales deeply. He also wishes he could get drunk, but still, a job’s a job.

“I guess we should check to see if everything’s good to go.”

The men head into the living room. The beamed, slanted ceiling gives it a mid-century vibe, but the staging muddles the aura. Jay remembers making the call to the staging company. “Give us your spares,” he told them, “Whatever it is you’re not gonna miss. Nobody’ll ever buy this house anyway.” 

The result is eclectic – a mix of office furniture, neo-Tuscan McMansion garb, and stuffy waiting-room lamps, all scattered atop popcorn-butter shag carpeting. Hideous, Robert thinks. Then there’s the ‘entertaining’ room, which is a particular pain in the ass to them, because the carpet was so disgusting, they had to replace it with that fake wood floor just to be able to stand being in there for more than five minutes. There’s a heady stone fireplace on one wall, the kind they don’t make anymore, a hearth. Next to it, equally hedonistic, a full bar. Through some doors, a red-painted room with a pool table and paintings of girls in fedoras on the wall. It’s all so cheap, really. Jay pulls out a folded piece of paper out of his jacket pocket along with a pen. He ticks some boxes and moves on.

The dining room’s the worst to Robert. Somehow the ugly floral pattern on the curtains stretches up in bloomer-like into a frilly cornice, carried through to the wallpaper and the ceiling, inescapable, suffocating. It smells like mothballs and old fabric. The whole house smells like that. 

The master bedroom’s the most normal – if anything in this house could be called normal. Mismatched art and staging furniture crowd blank walls. When someone comes into a house, Jay told Robert all those years ago, they should be able to picture themselves living in it. That’s the goal of staging. 

There’s two more bedrooms. The men go through them quickly. The first isn’t so bad – claustrophobic, but acceptable – but the saccharine pink tuille wallpaper of the second gives Renault a sympathetic toothache. The pair return to the kitchen to wait.


Both men are itching to check their phones, but there’s no point – there’s no signal in here, none whatsoever. Renault, cynical to the core, thinks about marketing the house to the anti-5G people. It’s unsettlingly quiet. The two men have no choice but to entertain themselves the old-fashioned way, through small talk.

“It’s really fucked up, when you think about it,” Renault muses.

“What is?”

“The house, Bob.”

Robert hates being called Bob. He’s told Jay that hundreds of times, and yet…

“Yeah,” Robert mutters, annoyed.

“No, really. Like, imagine. You’re rich, you founded a major multinational company marketing hairbrushes to stay-at-home moms, and what do you decide to do with your money? Move to Vegas and build a fucking bunker. Like, imagine thinking the end of the world is just around the corner, forcing your poor wife to live there for ten, fifteen years, and then dying, a paranoid old man.” Renault finds the whole thing rather poetic. 

“The Russkies really got to poor ol’ Henderson, didn’t they?” Robert snickers.

“The wife’s more tragic if you ask me,” Renault drawls. “The second that batshit old coot died, she called a guy to build a front house on top of this one, since she already owned the lot. Poor woman probably hadn’t seen sunlight in God knows how long.”

“Surely they had to get groceries.”

Jay frowns. Robert has no sense of drama, he thinks. Bad trait for a realtor.

“Still,” he murmurs. “It’s sad.”

“I would have gotten a divorce, if I were her,” the younger man says, as though it were obvious. It’s Jay’s turn to laugh.

“I’ve had three of those, and trust me, it’s not as easy as you think.”

“You’re seeing some new girl now, aren’t you?” Robert doesn’t really care, he just knows Jay likes to talk about himself, and talking fills the time.  

“Yeah. Casino girl. Twenty-six.”

“And how old are you again?”

“None of your business.”

“Did you see the renderings I emailed to you?” Robert asks briskly, not wanting to discuss Jay’s sex life any further.

“What renderings?”

“Of this house, what it could look like.”

“Oh. Yeah.” Jay has not seen the renderings.

“If it were rezoned,” Robert continues, feeling very smart, “It could be a tourist attraction - put a nice visitor’s center on the lot, make it sleek and modern. Sell trinkets. It’s a nice parcel, close to the Strip - some clever investor could make it into a Museum of Ice Cream-type thing, you know?”

“Museum of Ice Cream?”

“In New York. It’s, not, like, educational or anything. Really, it’s just a bunch of colorful rooms where kids come to take pictures of themselves.”

“Instagram,” Jay mutters. “You know, I just sold a penthouse the other week to an Instagram influencer. Takes pictures of herself on the beach to sell face cream or some shit. Eight-point-two million dollars.”

“Jesus,” Robert whistles. “Fat commission.”

“You’re telling me. My oldest daughter turns sixteen this year. She’s getting a Mazda for Christmas.”

“You ever see that show, My Super Sweet Sixteen? On MTV? Where rich kids got, like, rappers to perform at their birthday parties? Every time at the end, some guy would pull up in, like, an Escalade with a big pink bow on it and all the kids would scream.”

“Sounds stupid,” Jay says.

“It was stupid.”

It’s Robert’s turn to check his watch, a dainty gold Rolex.

“Fuck, still thirty minutes.”

“Time really does stand still in here, doesn’t it?” Jay remarks.

“We should have left the office a little later,” Robert complains. “The charcuterie is going to get –“

A deafening sound roars through the house and a violent, explosive tremor throws both men on the ground, shakes the walls and everything between them. The power’s out for a few seconds before there’s a flicker, and light fills the room again. Two backup generators, reads Jay’s description in the listing - an appeal to the prepper demographic, which trends higher in income than non-preppers. For a moment, the only things either are conscious of are the harsh flourescent lighting and the ringing in their ears. Time slows, everything seems muted and too bright. Robert rubs the side of his face, pulls back his hand and sees blood.

“Christ,” he chokes out. “What the hell was that?”

“I don’t know,” Jay breathes, looking at his hands, trying to determine if he’s got a concussion. The results are inconclusive – everything’s slow and fuzzy, but after a moment, he thinks it might just be shock.

“It sounded like a fucking 747 just nosedived on top of us.” 

“Yeah, Jesus.” Jay’s still staring at his fingers in a daze. “You okay?”

“I think so,” Robert grumbles. Jay gives him a cursory examination.

“Nothing that needs stitches,” he reports bluntly. Robert’s relieved. His face sells a lot of houses to a lot of lonely women and a few lonely men. There’s a muffled whine, which the two men soon recognize as a throng of sirens. Both of them try to calm the panic rising in their chests, to no avail.

“Whatever the fuck happened,” Jay says, trying to make light of the situation, “At least we’re in here. The bunker.”

Fear forms in the whites of Robert’s eyes.

“What if we’re stuck in here,” he whispers, afraid to speak such a thing into the world. The fear spreads to his companion.

“Try the elevator,” Jay urges, and Robert gets up, wobbles a little as his head sorts itself out, and leaves. A moment later, Jay hears him swear a blue streak, and from the kitchen window, sees him standing before the closed metal doors, staring at his feet. His pulse racing, Renault jogs out to see for himself.

“It’s dead,” Robert murmurs. 

“Whatever happened,” Jay says cautiously, rubbing the back of his still-sore neck, “It must have been pretty bad. Like, I don’t think we should go up yet. Besides, surely the office knows we’re still down here.”

“Right, right,” the younger man breathes, trying to reassure himself.

“Let’s just wait it out. I’m sure everything’s fine.” The way Jay says it does not make Robert feel any better. 

“Okay,” the younger man grumbles. “I’m getting a fucking drink, though.”

“Yeah, Jesus. That’s the best idea you’ve had all day.” Renault shoves his hands in his suit pocket to keep them from trembling.  


If you like this post, and want to see more like it, consider supporting me on Patreon!

There is a whole new slate of Patreon rewards, including: good house of the month, an exclusive Discord server, weekly drawings, monthly livestreams, a reading group, free merch at certain tiers and more!

Not into recurring donations but still want to show support? Consider the tip jar! 

Or, Check out the McMansion Hell Store! Proceeds from the store help protect great buildings from the wrecking ball.

We Interrupt This Broadcast to Bring You an Especially Cursed House

Hello everyone. Originally, this post was supposed to be devoted to the year 1978, however something came up, and by something, I mean this 2.2 million-dollar, 5,420 sq ft 4 bed/4.5 bath house in Colt’s Neck, NJ. 

You see, usually, when a listing goes viral, I’m content to simply retweet it with a pithy comment, but this house genuinely shook something in me, genuinely made me say “what the (expletive)” out loud. It is only fair to inflict this same suffering onto all of you, hence, without further ado: 

Looks normal, right? Looks like the same low-brow New Jersey McMansion we’re all expecting, right? Oh, oh dear, you couldn’t be more wrong

Guess who’s making a list and checking it twice? 

Guess who’s gonna find out who’s naughty or nice?

Guess who’s coming to town? 

Guess who’s coming to town to drag your ass into hell?

A gentle reminder that it is not yet Thanksgiving. 

But oh. Oh. It continues:

If you’re wondering what’s happening here, you’re not alone, and sadly there is no convenient way to find out via a kind of haunted house hotline or something. 

I can’t even label these rooms because frankly I’m not even sure what they are. All I am sure of is that I want out of them as soon as humanly possible. 

r̸̘̆e̴̝̻̽m̵̡̼̚ȩ̵͑̎ͅm̷͍̮̉b̸̥̈e̶̯̺̽͗r̸̝͊͠ ̸̡͎̅̀t̴̯̲̓ȯ̷̮̫ ̷̜̅̀ŵ̶̟̱ā̴̭̘s̸̥͋h̴͉̿ ̵̡̑y̸̩͈͑o̷̹̭͛͝ů̷̩̮̔r̶̜̃ ̴̠̗͋ẖ̴̈́͛a̸̢̟̐͒n̶̩̟̆ḍ̵̍̀s̴̨̈́

How is it that a room can simultaneously threaten, frighten, and haunt me? Me, of all people!

My eyes do not know where to go here. They go to the window, they go to the fireplace, they go to the massive mound of fake plant and statuary currently gorging on the leftmost corner of the room, they go to my hands, which are shaking. 

“Hello, I would like to get in touch with the Ministry of Vibes? Yes, I’ll hold.” 

I haven’t been this afraid of a shower since I went to Girl Scout camp in the fifth grade and there was a brown recluse spider in the camp shower and I screamed until the counselor came in and told me it was only a wolf spider but it turns out those still bite you and it hurts

I love watching Still Images on my Television Set :)

Nobody make a sound. He’s watching you. 

i spy with my evil eye:


Their souls are trapped in these photographs forever :)

Okay, phew, we made it out alive. Here’s the back of the house I guess. 

Well, I hope you’re as thoroughly disturbed as I am. Seriously, I’m going to have trouble sleeping. I mean, I already have trouble sleeping, but this is just making that existing problem so much worse. 

If you like this post, and want to see more like it, consider supporting me on Patreon!

There is a whole new slate of Patreon rewards, including: good house of the month, an exclusive Discord server, weekly drawings, monthly livestreams, a reading group, free merch at certain tiers and more!

Not into recurring donations but still want to show support? Consider the tip jar! (Tips are much appreciated since I am making a cross country move in two weeks!!!)

Or, Check out the McMansion Hell Store! Proceeds from the store help protect great buildings from the wrecking ball.

The McMansion Hell Yearbook: 1977

Howdy, folks, and happy October! (It’s snowing here in Chicago lol) 

Before I get down to business on today’s post, I want to let you know of two big events coming up this week: 

First, I’ll be in conversation with Susan Chin and Vinson Cunningham tomorrow evening (10/28) to talk about urbanism during the pandemic (virtually) at the Museum of the City of New York. More info and tickets here

 I am giving this year’s Brendan Gill Lecture in Architectural Criticism at Yale via Zoom on Thursday the 29th of October at 6:30 Eastern Time. Admission is free. Here’s a link to the talk which includes info on how to register. 

Alright, now back to the main event. We’re back in Cook County, Illinois because of course we are, and this house falls into the rare McMansion Hell category of “this house is terrible but also kind of cute somehow????” 

It’s a shame you can’t really see the turret because it adds so much. Anyways, this house is peak 70s McMansion: longer than it is tall, involves a mansard, big picture windows, not too adventurous roof-wise. Still, it’s 6900 square feet boasting 5 bedrooms and 6.5 baths all at a whopping $1.5 million dollars. Just some pocket change, you know…

Let’s see inside, shall we?

Lawyer Foyer

All I want is some Looney Tunes action where some’s coming up from the basement and someone’s coming in the front door and WHAM!!!! 

Sitting Room

I kind of stan the dog pots though… 

Dining Room

I think the wallpaper might be crabs???? (????)


Pros of tile countertops: v twee and cute
Cons of tile countertops: grout 

Also we NEED to bring back the kitschy farmhouse aesthetic from 40 years ago. No more quartz countertops. It’s time for tiles with chickens on them!!!


Is this room supposed to be like weirdly tropical?? or Parisian??? or Martha Stewart??? or???

Vibe check: [please calibrate vibe checker and try again]


After all, inside every middle manager is a languishing Hemingway…

Main Bedroom

“Struggled hard for these views (six arm flexing emojis)”

Also, disclosure: McMansion Hell will no longer use the term “master bedroom” because it’s antiquated and never made much sense after the (American) Civil War if you really think about it for more than three seconds. 

Main Bathroom

where to purchase malachite wallpaper asking for a friend (the friend is my office)

Spare Bedroom

Nothing says “I am a fun-loving carefree and slightly cRaZy girl” like this font:

Alright, that’s it for the inside. Instead of the rear exterior though, I’m going to end this post with a fun aerial shot instead just to show that my suspicions about this house have been confirmed. 

Secret Aerial Footage (helicopter sounds)

See, this house is actually very weird!!!! It is not as cute when all of its wily tricks have been revealed!!!!

Okay, that’s it for 1977. Stay tuned for 1978! 

If you like this post, and want to see more like it, consider supporting me on Patreon!

There is a whole new slate of Patreon rewards, including: good house of the month, an exclusive Discord server, weekly drawings, monthly livestreams, a reading group, free merch at certain tiers and more!

Not into recurring donations but still want to show support? Consider the tip jar! (Tips are much appreciated since I am making a cross country move in two weeks!!!)

Or, Check out the McMansion Hell Store! Proceeds from the store help protect great buildings from the wrecking ball.

Public opinion has softened its views on Brutalism. That isn’t enough to stay the wrecking ball.

Public opinion has softened its views on Brutalism. That isn’t enough to stay the wrecking ball.:

I’m back in The Architect’s Newspaper, where I’m talking about my favorite subject (Brutalism) and my least favorite subject (capitalism).

The McMansion Hell Yearbook: 1976

Howdy, folks! Today’s house comes to us from my newly adopted county of Cook County, Illinois, and boy can this baby fit so many 70s house stereotypes in it. 

It’s got everything: weird spanish colonial revivalism, an external layout that can only be described as post-split-level, a 3 car garage, and it’s brown! This lovely 5 bedroom, 5 bath 5200 square foot estate is relatively affordable by McMansion Hell standards, coming in at around $600,000. There’s a lot of house to cover, so let’s get the ball rolling! 

Lawyer Foyer

This foyer has all the elements of a contemporary lawyer foyer (large chandelier, grand staircase, two stories) except for the oversized transom window over the front door. The fact that the house looks like a split-level on the outside is interesting because it’s a regular two-story house on the inside, furthering the hypothesis that the Lawyer Foyer itself is an offshoot of the 1.5 and two-story entrances present in split levels. In many ways this house is a transitional example nestled between two eras: the split-level/ranch of the 70s and the two-story neo-eclectic houses that would become popular in the 1980s. 

Sitting Room

Fun fact: a look at recent IKEA catalogs demonstrates that the grandma couch is slowly wedging its way back into America’s living rooms. 

Dining Room

I am weeping with envy at those chairs. (Instagram story vagueposting voice) Some people just don’t know what they have. 

Living Room

The overstuffed leather sofas might not be pretty but they are authentic.


Why I hate the kitchen island/peninsula stovetop recapped:
- can’t use the island for seating bc cooking stuff is hot and steamy
- one casual lean and you’re burned
- no backsplash to catch like overboiling pasta sauce
- wastes valuable counter space

I can go on.

Master Bedroom

Personally if I had all that extra space in my bedroom I’d put something cool like a pool table or a hot tub in there bc why not???

Speaking of tubs…

Master Bath

One must wonder why brown bathroom fixtures exist in the first place because frankly it’s not a very flattering color considering the functions. Let’s just say it was a different time. 

Bedroom 2

As someone who grew up in the era of Toyota Corolla hegemony, 70s cars are extremely funny to me - like they take up half a block and get 4 miles to the gallon??? No wonder there was an oil crisis!!! 

Bedroom 3

The virgin midcentury modern collector vs the chad grandma using a 1967 teak Dunbar sideboard as a display case for their doily collection 


This whole post is a ploy to get the zoomers to watch Cheers

Alright folks, our little house tour has come to a close - it’s time for our favorite part:

Rear Exterior

(looking enviously at other countries with functioning governments beginning to open back up): yeah ok you do you, i’m just gonna watch the tour de france in a bathrobe and rank the teams based on how cancelled their sponsors are.

Well that does it for 1976! Join us soon for another installment of the Brutalism Post and keep your eyes peeled for whatever wretched house the year 1977 has bestowed upon this cursed land. 

I know that these are economically uncertain times, but many creators including myself depend on Patreon for most of their income, so if you have a minimum of $12/year to spare and are into bonus content, then do I have some good news for you:

If you like this post, and want to see more like it, consider supporting me on Patreon!

There is a whole new slate of Patreon rewards, including: good house of the month, an exclusive Discord server, weekly drawings, monthly livestreams, a reading group, free merch at certain tiers and more!

Not into recurring donations but still want to show support? Consider the tip jar! (Tips are much appreciated since I am making a cross country move in two weeks!!!)

Or, Check out the McMansion Hell Store! Proceeds from the store help protect great buildings from the wrecking ball.

The McMansion Hell Yearbook: 1975

Howdy, folks! We’re halfway through the 70s, and I thought I’d celebrate with a time capsule house stuck weirdly enough, in the 80s. Our house this time comes to us from Fairfield County, Connecticut, and while it may not be an obvious contender on the exterior, I promise you won’t be disappointed once we head through that door. 

This house, despite its modest exterior, boasts 4 bedrooms, 4.5 bathrooms, and just over 5300 square feet. It can be yours for just over $2.2 million USD.  I know you’re dying to see what’s inside, so I won’t keep you any longer.

Lawyer Foyer

As you can see, painting the walls white did not take the 70s out of this house. The disappointing part is that this is the room with the most vestiges of its 70s past - that wrought iron railing, pink linoleum, and pseudo-gothic chandelier definitely affirm that originally this house was much, much groovier before its 80s redux. 

Great Room

The realtor described this house as “transitional” which in some cases is a polite way of saying “trapped between stylistic movements and terrified to death of choosing one.” 

Sitting Room

Alright, alright, here’s one for the 80s aesthetic blogs. You’re welcome. 

Dining Room

As a form of economic stimulus, I am willing to accept giant cabinets and twee bird knickknacks. Speaking of giant cabinets, that one is, like, hearse-sized. How many candelabras and cloth napkins could one family possibly possess? 

Also, for some reason, the listing did not include any pictures of the kitchen, so we’ll have to go right into the master bedroom. 

Master Bedroom

Even in the 80s, was there ever a time where this aesthetic didn’t look, well, grandmotherly?

Bedroom 2

I’m moving in a few weeks and my back hurts just thinking about trying to lift that furniture!!!!

Bonus Room

I have to give credit where credit is due: this room is cool, and I would absolutely chill in it. Which goes to prove how deeply uncool these rich people are for not using it for chilling or any other activities. 

Rec Room

The drop-ceiling/can light combo is somewhat rare in terms of McMansion bonus rooms, as is that diagonal wood paneling which I unironically stan. Forget shiplap!!!

Alright, that’s it for our interior. Now to check out the rear exterior which proves once and for all that this house is, in fact, a McMansion. 

Rear Exterior

Honestly, I don’t know what kind of house this is - my guess is that it’s, like, a post-split-level, whatever that means. Either way, it’s super tacky and I’m glad I found it so I could share it with all of you. Check back here soon for another 70s house, as well as a much-needed update to the Brutalism Post. 

I know that these are economically uncertain times, but many creators including myself depend on Patreon for most of their income, so if you have a minimum of $12/year to spare and are into bonus content, then do I have some good news for you:

If you like this post, and want to see more like it, consider supporting me on Patreon!

There is a whole new slate of Patreon rewards, including: good house of the month, an exclusive Discord server, weekly drawings, monthly livestreams, a reading group, free merch at certain tiers and more!

Not into recurring donations but still want to show support? Consider the tip jar! (Tips are much appreciated since I am making a cross country move in two weeks!!!)

Or, Check out the McMansion Hell Store! Proceeds from the store help protect great buildings from the wrecking ball.

#HousingLIVE Join The New Republic’s Kate Wagner for a special after-hours live action version of her satirical McMansion Hell blog. July 14, 2020 at 5.30pm EDT.

#HousingLIVE Join The New Republic’s Kate Wagner for a special after-hours live action version of her satirical McMansion Hell blog. July 14, 2020 at 5.30pm EDT.:

Howdy! Join me at the NewCities New Housing Solutions conference (along with much more important people like Ilhan Omar) where I’ll be roasting buildings and raising money for Moms4Housing! Link to submission and registration above. 

Design in Dialogue - Exhibitions - Friedman Benda

Design in Dialogue - Exhibitions - Friedman Benda:

Howdy folks! Join me on Design in Dialogue tomorrow (Monday, June 15th) at 11AM EDT for a (Zoom) talk on the best and worst impulses in contemporary architecture. More info and RSVP in link. 

How Normie Minimalism and Farmhouse Chic Took Over Contemporary Design

How Normie Minimalism and Farmhouse Chic Took Over Contemporary Design:

Hello! I wrote for Hyperallergic about how minimalism went from high design to normie chic.