The tests passed. The extension was still broken.

Building it reminded me of something I already knew from doing QA professionally. A green test run feels great, especially after you just spent hours fixing something annoying. But it can also give you a completely false sense that everything is fine.

The tests aren't really lying. They proved exactly what you told them to prove. You just forgot half the questions.

passing tests dont mean the product works

The first version of ChannelFence had a simple job. Find the creator attached to a video, add a block button, and hide that creator afterward. Pretty simple, at least on paper.

Then it has to work on the actual YouTube website.

Youtube is not one page with one reusable video card. It has the Home feed, search layouts, recommrndation rails, channel tabs, playlists, comments, compact Shorts cards, the Shorts viewer, promotional shelves, collaboration cards, and random experiments that not every user even gets. It also acts like an app instead of a normal website. Elements load late, get removed, get reused, and change without the page doing a real refresh.

I could write a test for one video card, watch it pass, then open youtube and find a completely different card where the button never showed up.

The test wasnt useless. It was just a very incomplete version of the product.

you start avoiding your own bugs

People who use the same software every day build habits around it. Developers do this really fast because we already know how the feature is supposed to work.

You click the part of the card you know is reliable. You wait an extra second because you know the page loads slowly. You refresh after changing the extension because thats normal during development. You avoid the layout that broke last week without even realizing your avoiding it.

Eventually the workaround doesnt look like a workaround anymore.

A new user has none of that context. They click the obvious button. If it does nothing they click it again. If it still doesn't work, most people are not going to write a nice detailed bug report for you. They uninstall it and move on.

So from my side there might be no reported bug. From their side the extension just didnt work.

stuff our first tests completely missed

One bug put three or four ChannelFence block buttons on the same channel page. The extension saw several nested youtube containers and treated every one of them like a seperate place that needed a button.

The feature technically worked. The page looked stupid.

Another bug was a lot worse. On some channel pages ChannelFence picked up unrelated text like "Community" instead of the creator. On other pages it saved the channels current display title instead of the stable youtube handle. So the user clicked Block beside one creator, but the extension saved the wrong label or saved a name the creator could change later.

Collaboration cards were another fun one. A video can have two real creators attached to it, which means grabbing the first name you find is not good enough. Both controls need to point at the right stable handle without one creator getting mixed up with the other.

Then there were Shorts.

Blocking one creator in the Shorts viewer worked in a basic test. Blocking several in a row was where it got buggy as hell. The same Short could stay on screen, the viewer could lose its controls, navigation could happen twice, or the user could randomly get sent back to Home. Sometimes a blocked Short would show up again after reopening or refreshing the Shorts page.

Those were all slightly different timing and state problems. None of them showed up in a test that opened one Short, clicked once, and checked that a block record existed.

Performance was another blind spot. Scanning a small feed with ten blocked creators felt fine. That tells you basically nothing about a person who has blocked 500 or 1,000 of them. Rebuilding the same matching data or rescanning the whole page too often can make ChannelFence slow, and worse, make youtube's own buttons load late too.

The extension didnt just need more happy path tests. It needed tests that were rude to it.

what changed after finding all of that

Every embarrassing bug should turn into a regression test. Otherwise you fixed the bug but didnt really keep the lesson.

The duplicate button bug became coverage for nested channel layouts, client-side navigation, and actual separate collaborators.

The identity bug became coverage for stable youtube handles, hidden header labels, display names that can change, and old saved entries that still need to work after an update.

The Shorts problems became tests for stale creator data, controls that load slowly, unavailable creators, double navigation, reloads, and a bunch of blocked creators in a row. One deterministic test uses a simulated queue of 50 consecutive blocked creators and makes sure the viewer eventually reaches an allowed creator without freezing or falling back to Home. A separate live test uses a real YouTube Short with a 50-entry block list, because a perfect local fixture can still be wrong about the real site.

Large block lists now get tested too. There is coverage with hundreds of entries in the settings page and 1,000 blocked creators while matching feed cards. The extension indexes the values it matches against, doesnt rebuild the same unchanged list for no reason, and tries not to do work inside a tab you cant even see.

This is probably my favorite part of automation. A bug teaches you something once, then the test keeps bothering the code about it forever. I don't have to remember every weird youtube layout manually for every release.

manual QA and automation aren't enemies

People frame this as manual testing versus automation alot and i don't think its a useful argument.

Automation is great at repeating things we already know about. It can cover dozens of layouts, data combinations, and state changes every time the code changes. It can catch an old bug coming back before i even load the extension myself.

Exploratory testing is how a lot of those scenarios get found in the first place.

A person notices that a button is technically there but almost impossible to see. They ask why blocking from one menu behaves different from blocking beside a creator name. They scroll, reload, go backward, open another tab, use a stupidly large block list, or click before the page is ready. They can see that an assertion passed while the actual experience still sucks.

You need both. Manual QA expands the idea of what could go wrong. Automation stops that idea from shrinking again six months later.

developers should own quality, but that doesnt replace QA

Developers should test their own work. Obviously. Having a QA person shouldnt be an excuse to hand over unfinished garbage and call it ready.

But "developers own quality" sometimes turns into "developers repeated the same checks they already did while building it." Those aren't the same thing.

The value of QA isn't that a tester knows how to click a button better. Its that their job is to question every assumption around the button.

What happens before it appears? What if it appears four times? Which creator identity does it save? Does it still work after navigating without a refresh? What if there are two creators? What if the page is translated? What if the person already blocked 500 channels? What if youtube changes the container but the page still looks almost the same?

Those questions take product knowledge, technical knowledge, and time. If nobody is actually responsible for asking them, saying everybody owns quality doesnt magically make it happen.

building and testing the same product is weird

ChannelFence is a small project, so im both the developer and the QA engineer. That makes this problem pretty obvious.

When I'm writing the code, I know what I meant. When i test it properly, I have to stop giving the code credit for my intentions.

I have to treat it like something a stranger found in the Chrome Web Store. They don't know which youtube layouts are difficult. They dont care that a race condition only happens during client-side navigation. They dont care that the creators handle was missing for half a second while the page loaded.

They care if clicking Block made that creator go away.

Thats the standard the product has to meet. Not whether the implementation was clever, whether the demo worked one time, or whether every test we happened to write was green.

The tests can pass and the product can still be broken. QA is the work of finding the distance between those two things, and sometmes that distance is huge.

Well, I'm just a senior QA engineer writing apps for fun after work, so maybe im overthinking all of this. But if one tiny extension can find this many ways to be broken while every test is green, I dont think the giant products we use every day are magically safer. Maybe somebody just stopped asking annoying questions.