It's been a while...

Haven't updated here in quite some time now, simply haven't had the time for it. I've been completely swamped at work, trying to make as much as possible of our ridiculously unrealistic release plan for the fall/winter. I didn't want to update the blog in my spare time either, after the "breakdown" earlier I had to shut off all things work just to get back to my old self. It feels a lot better now, except for a "stress-stomach" which, I've learned, I just have to learn to live with. Thanks a lot, what a nice bonus...

After a lot of nagging I finally managed to get an intern to my department, if only for three months. No previous test experience (except for open beta-testing) but with a lot of gamer experience, which actually is very useful. I hope it will work out, the main reason for getting an intern at this time (not that we don't need one ALL THE TIME) is that I'm going on paternal leave for December. Yup, the family is growing, expected delivery within a few days. Exciting indeed!

We had a disaster of a release earlier this week. From the very beginning, I warned everyone involved that this is a very critical release with extremely high impact on our users and therefore it has to be thoroughly tested for an extended period of time. My request was unfortunately denied, somehow the release date was more important than quality. The result? As I said, complete disaster. Nothing worked according to plan and several of the issues were things I already raised my warning flag over. To make things really perfect, I couldn't even be there for the chaos since I had to stay home with my sick child. All I could do was to watch from the outside, and on the forums follow the fiery discussions about bad testers etc. If they only knew the truth behind the story... Sure, I partly blame myself for not making the "people in charge" change their mind about the release but at the same time I don't know if that would've been possible in the first place. They still have very little understanding about test and I do believe that they confuse testing with checking. Oh well...

Sometimes it sucks to be me

I know I've been late with updates here but no excuses this time. I've been forced to cut down on activities for a while due to health issues. Fortunately I realized what the symptoms actually were in time, before it got even worse.

What's wrong with me? Well, it seems that I don't say "No" enough. That, and the fact that I take a lot of pride in my work, has been one of the reasons I finally hit the wall, stressed out to the max. But I will solve this, don't worry. I've already quit one very time consuming extracurricular activitiy - I'm no longer a board member in the housing cooperative. Neither will I spend extra hours at work just because management doesn't listen to our status reports and/or warnings. My primary focus is now my family and myself.

So, other than that, it's pretty much business as usual. The participant community still whines a lot, not seldom without good reason but ever so often about things they just don't understand (not calling them dumb or anything, they just don't have all the info). Management still don't fully understand what testing is all about and I find that my colleague and I still are expected to work miracles.

There were some good news for the company last week though, which potentially could mean a lot of improvements for us. But I'll try not to get too excited before I actually can see those improvements with my own eyes.

Back in Black...

...black being the color of my mood. This is just not funny anymore. After the vacation I was actually quite happy about going back to work, for some reason thinking things would have changed to the better. But no. Hell no.

Test is still an underappreciated part of the organization, a necessary evil, that management has decided to ignore as much as possible. When looking at the release plan for the fall, it becomes painfully clear that they either have no idea how much work we have to do for each massive update and leaves us with no more than five days to test, or they simply don't care if everything is tested or not, as long as we meet the release date.

To paraphrase Darth Vader: "I find the lack of professionalism disturbing".

I've actually reached the moment where I only wait for the upcoming, already paid for, conferences. As soon as they are over, I will do everything to get away from this underappreciated, extremely underpaid, job and find my luck somewhere else. The current situation is simply not good for me.

Charging the batteries

I know that I haven't been very active here lately and I apologize for that. This spring has been nothing but chaos at work and my motivation recently reached a new All Time Low. Fortunately for everyone involved I'm now on a much needed four week vacation, time that will be spent re-charging my batteries and hopefully my motivation as well.

This morning I read a tweet from James Bach, pointing me to a blogpost by Parimala Shankaraiah, the "Curious Tester". In this post she writes about interviewing/recruiting testers and that got me thinking about the time we recruited my colleague, a little more than a year ago.

For a long time, I had been the Lone Tester (Hi-yo Silver, away!) at the company, constantly swamped with work. As we were gearing up for a major, and I mean Major, platform update that summer we really needed to expand the test department. If not only for the increased scope of the update then because of the fact that I was going away on paternity leave. You may think that was extremely bad timing, but the fact is that the original release date for the update was before my planned leave (which I booked six months earlier). But the update was postponed, and rightfully so.

Around the same time as I booked my leave I also flagged to management that we were in desperate need of another tester. A wanted ad was posted and I went through a lot of applications, picking out quite a few interesting ones. But for some reason management didn't want to hire at that time after all, and it wasn't until a month before my leave that they realized they were about to become "testerless", right before the update! So a new wanted ad was posted and we had to rush through the whole recruiting process.

Again, I read a lot of applications, threw away quite a few right away due to the fact that they had misunderstood the actual job (and location for that matter). I can't blame anyone but our own HR department for that though, the ad was very vague. Since our product is a Virtual World, or a "game", many people thought we wanted beta-testers who could work from home. This was not the case, obviously.

We (the Development Manager and I) picked out a few interesting applicants and called them in for a quite informal interview. Several of them had some decent testing experience but "failed" (harsh word, but I'll use it in lack of a better one) in other areas, like not being interested in games etc. For me, the personal chemistry was a very important thing since I was supposed to work close together with this person every day.

To be honest, it didn't look good. But then a friend of mine gave me a tip about one of the applicants they had interviewed for a Customer Support position. While he wasn't chosen for that position, they felt that he had "something" and thought that he could be of interest. I said "OK, ask him to send in an application asap".

The application was on my desk when I came in to work the next day. This was a young guy with a lot of experience in MMORPG:s, playing and beta-testing. No other testing-experience, but man was this guy thirsty for knowledge! He wrote that when he got the call from my friend he didn't really know if he was interested at first but googled for "Software testing" to find out what it's all about. He then described how he lost himself in blog after blog and how he definitely wanted to be a part of this world. So we called him in for an interview...

You know how I mentioned personal chemistry earlier? Well, this was it. I quickly realized that this guy shared my values, ethics and sense of humor. He was also very well-versed in MMORPG:s and usability and had an honest interest for the work of a Software Tester. It was decided right after the interview that this young man, if he so desired, was to be my padawan...

He said yes, fortunately, and was literally thrown right into a pile of work. He's been doing an excellent job from day one and we work really well together. Hell, hadn't it been for him I'd most likely be burnt out today.

Release problems - a real test on the nerves

So, we released quite a major update the other day. As usual, I didn't feel that we had enough time to test everything (when do one ever have enough time?) but even so I felt quite satisfied when we shipped.

Didn't last long though, the following night the whole production environment went down for several hours straight! When I came in next morning I was pretty nervous that this very serious issue was caused by something we missed during testing, but after a few hours of investigation it was stated that it was caused by a hardware failure. As soon as it was found it was easy fixed and won't happen again. Most important (for our self esteem anyway), it wasn't our "fault".

Of course, there's been some other issues as well, but it looks like they're mostly caused by design errors. Several of them were questioned by us during testing, but we were assured that "that's the way it's supposed to be". The outcry from the community seems to have changed their minds though...

On a sidenote: just read the new presentation of the new iPhone 4. I must say that I was only mildly impressed. Sure, the new screen looks great but the camera is still sub standard. Video chatt? Let me tell you something, I rarely want to see the ones I'm talking to. And if they could see me, I wouldn't be able to do faces whenever they say something stupid ;) The data transfer rate is clearly designed for the American market (slower than ours). But hey, the release of this version will mean that the 3GS version will become even cheaper and I will definitely pick up one of those.

Certified? Yeah, so what?

In his two latest blog posts (here and here), James Bach again talks about ISEB/ISTQB certifications and how bogus he thinks they are. I couldn't agree more. Sure, I am ISTQB certified myself, but even before I attended that three day course (which for me turned out to be a two day course, being sick the first day) I knew that what was going to be presented to me wasn't the truth. Well, not the ONLY truth anyway.

I learned some things from this course, I admit that. But there were also other things that felt completely outdated and irrelevant. Nonetheless, I'm happy I attended (and even happier I didn't have to pay for it myself) since I'm of the opinion that lessons can be learned anywhere and everywhere. I like to read and learn about many different techniques, approaches and methodologies and mix them together where I see fit. Take a few goodies here and a few goodies there...

The inherited burden of test cases

My predecessor at work spent a lot of time creating test documents for each and every system we have, documents filled to the brim with different test cases. When I first begun working as a tester (and later on as test lead) at this company I found these test cases very well written and useful and they were very self-explanatory, making it easy for anyone (well, almost anyone) to step in and help out with the testing.

However, the times have changed. Since our big conversion from one graphics engine to another and the many design and functionality changes it brought to the table I've found the old test cases to be much too specific and just a pain in the a** to keep updated. And if they're not updated, they're useless.

So, I decided that some major changes had to be done and there were in reality only one way to go: Exploratory Testing. We will still have to use test cases to some degree, but with short descriptions of the scenarios, some expected results (not specific figures, but the formulas used to calculate the correct results) and information on where in the system to find these results. What used to be a 10 page document filled with test cases will now be one, maybe two pages long. A lot easier to manage and keep updated, and not as scary to look at.

There's only one problem; Finding the time to redo all these documents. We're under a extremely heavy workload and are already cutting corners to meet the (not so realistic) deadlines set by management. It doesn't help that we're currently sitting in an open landscape office, constantly being disturbed by everyone and their grandmother. But that's another story...

The documents will be updated and used as a starting point for our exploratory testing, that much I can promise. But when? Soon, I hope...

Setting things straight

Among the community of our users there are very high-voiced opinions about the quality of each release. Nothing strange about that, would be much weirder if they didn't care. However, there seems to be a tendency to always, ALWAYS, blame the testers when something doesn't work like it should (or maybe it works as intended, but it's not to their liking). Therefore I'd like to set some things straight:

1. The deadline.
We have a deadline for each release that has to be met. Usually, the Project Managers plan one week of testing (which, in my opinion isn't enough) but even if the developers fall behind our test week does not get extended, which means that we have to work miracles in order to test at least most of the things during just a few days.

2. Resources.
We are a small test department, very small. We also do not have all the different hardware setups we would like and therefore can't catch all hardware related issues that may appear. We are working on it though, trust me.

3. Go/No Go decision.
This one is important. I, as Test Lead, DO NOT decide whether a release is ready to be shipped or not! What I do is to give my recommendations and professional opinion regarding the current status of the release, it's then up to the Project Manager to decide whether to ship it or not. Believe me, my recommendations have been overlooked many, many times.

I'm not saying that our test department is perfect, not at all. I just want to point out that we do the best we can under the circumstances. I have an agenda to improve our test process, but it will take time. I'm not giving up yet though.

OK, here we go...

Who am I? Well, I work as inhouse Test Lead/Tester with a Virtual World developer, have done so for a couple of years and will (perhaps) keep on doing it in the future.

I'll try to update this blog if not daily then at least on a weekly basis. I will NOT go into too much detail of my work (partially due to the NDA I've signed, but also simply because I don't want to) but I will post general thoughts and questions that can be found in my brain, regarding testing and other topics.