30 Days of Agile Testing! Day Five.

Day 5:
Pair with a developer on a feature

I pair with our developers to test stuff all the time! It’s a great way to work, and I love talking about it, but I kind of forget that in some environments it’s not that normal.
So - what you’re about to read is an experience report of a normal session of pairing with a developer to do some testing. We're not at a super interesting phase in our project, so apologies if it's a bit dry, but it's how it went down!

This is Ben, and we’re testing some PAN sanitisation together.


I promise you, he's thrilled I took this photo.

Context: what is PAN sanitisation?

PAN stands for Primary Account Number, essentially, a Credit Card number.

PAN sanitisation is something we do to ensure customers can’t accidentally save their Credit Card numbers in places they shouldn’t. It’s a security feature, to prevent people from doing silly things.
As an example, if they were to put a PAN (or a value that looks PAN-like) in an address field, we would prevent that:


What we tested

We’ve built a new feature - without going into detail, it’s a payment screen.
Ben has just added PAN sanitisation to this new screen, and we’re going to test it together, locally, on his machine.

Our first task was to check the validation was working where expected. You know, the happy path.
This meant entering PAN-like values into fields such as ‘notes’ and ‘payment reference’ and ensuring they couldn’t be saved.

So far so good.

We did discover a little side mission though - there’s a ‘quantity’ field that validated correctly, but also has a max of 1000 units. We should really limit that field to four characters, so we made a note to raise a ticket for that later.

Another thing we wanted to be sure of, was that we didn’t validate this when we weren’t supposed to.
Imagine if the Credit Card field wouldn’t allow a PAN-like value - nobody could ever pay us.
So we checked that we could still make payments using Credit Cards successfully.

We could!

We also wanted to check other fields that accept values that could look like PANs. For example, in some countries a phone number could look a bit like a PAN.

These fields were also good!

In stepping me through his solution, Ben mentioned that we do PAN validation on both the client side and server side. This is why talking the solution through with your developer is so important - if we hadn't had this conversation, I wouldn't have known!
Anyway, this is a precautionary measure - if anyone managed to somehow slip past our client side validation, they would still be protected. Double security!

At the UI level we could only test the client side validation - but because we were testing on Ben’s local environment, we could easily turn the client side validation off - and watch the server side validation do it's thing.

The server side validation worked - the fields couldn't be saved.

But - the validation messages coming from the server could have been a bit better - there’s a small improvement to be made here, Ben is going to change it before we ship this.

Finally, it was my turn to try and come up with anything we may not have covered. The only thing we'd missed was a 'user details' popup, that was present, but not an obvious on screen.
It’s a shared component, so our expectation was that it would already be sanitised, but we needed to check.

All the fields in this screen validated against PANs as we expected!

And that’s it! That’s about forty-five minutes of pairing with a developer on a new feature. We didn’t find many problems, because Ben’s really good, but that’s how it works.

Thanks for reading!

- JE