Quote:

 

The forums have been bustling with questions recently regarding the process of QA with projects, and why some projects have bugs in them. Well, hopefully, this Developer Blog will explain the process of QA a little, so that you can better understand how content takes the step from a design proposal to being enjoyed by hundreds of thousands of players worldwide.

The step into QA

When a piece of content comes into QA, it typically takes no more than few hours to find errors, glitches, bugs and concerns. Worry not, though, as this is expected and it’s our job to discover these things! From this moment to the content being released, we work through specific test plans on testing each element of the content, known trouble areas and general exploratory testing.

Any issues found are then sent to the designated developer for fixing, be it anyone from a graphics artist to a content developer to web content. An example of the scope of issues found in a project: an upcoming quest currently stands at 123 bugs, has been in QA for one week and is being tested thoroughly to get it released in time to be enjoyed by everyone.


More than breaking things

While a large portion of the task at hand is to break and destroy content sent our way as much as possible, which can be extremely enjoyable and equally frustrating at times, we are also very opinionated and believe we understand the players.

When a piece of content comes into QA, it is inevitable that opinions on how to improve design decisions or gameplay mechanics will fly about, trying to make the content more appealing to players. This can be anything from making a weapon look more deadly and suited to the style of RuneScape, to simplifying or increasing the difficulty of puzzles to keep the players engaged (but not bored or frustrated), and even completely redesigning core gameplay mechanics.

These feedback decisions are then relayed to the developers, and with the projects being their own, the decision rests on their shoulders. If the QA team feel particularly strongly about a decision, the concern can be raised to the leader of the particular department, who will weigh up the feedback given, the developer’s feelings, and the most important variable of all: time.


Priority

Time is extremely important when working on any MMO, and even more so when that MMO receives updates on an almost weekly basis (as opposed to the more typical three- month cycle). It’s rather remarkable, actually.

Unfortunately, given the strict schedule for release the game has, priority has to be put in place for what can be done in the time allocated. Being given two weeks to fully QA a quest, for example, means that the QA team working on it have to determine where to allocate time, as in those two weeks, the quest has to be tested twice - once in the work in progress (WIP) test environment and once in the release candidate (RC) test environment.

Further on from this, to maintain such a quick turnaround on releases, the QA team typically has anything from four to ten projects on at once, split between the team (which also have to be balanced alongside any fixes for previous projects or improvements that could not have been implemented in time), as well as fixes for bugs found by players and relayed to us via the Bug Reporting Forums or Bug Report feature on the homepage (we read and investigate every report!).


Why is content broken?


There are typically three core reasons why a piece of content contains bugs at release:

Time
This applies more to game ‘features’ than outright bugs. We know of a feature that might cause issues, but, unfortunately, the time it would require to try out and test alternatives may be longer than is available for the project. For issues such as this, we start to consider changes straight after the content is released and wait to see what feedback the players give, so we know the best direction to take. A good example of this is Mobilising Armies, which we’ve recently tweaked to iron out some issues players have had with it since launch (e.g. the number of briefing rooms).

Test Environments
As detailed above, and in even further detail by Mod Nexus in his ‘Upload Manager’ Developer Blog, the content has to be tested in both WIP and RC. WIP is the core environment that all developers use to create content for RuneScape, be it for an update scheduled for the next week, or one for next year.

Even a small change in WIP can have a large impact on the game. It is not uncommon for one change to affect just a single item or feature from a selection of millions - something that is nigh on impossible to locate even if given a year to find. Not only this, but when the piece of content moves from WIP to RC, it’s like moving content from a copy of RuneScape that is six months in the future to current-day RuneScape. The differences can easily break content, requiring us to quickly discover what, where and how this has been caused - as this move into RC means the content will go up as the next update, the clock is ticking.

Bugs Missed
Given the scope and scale of the game, along with the constant progression of updates, bugs are occasionally missed. Only a small percentage of bugs that actually fall into this category?

When you consider that, for example, there are 713 head items, 211 cloaks, two graphical modes and 12 graphical settings, that creates 3,610,632 possible combinations. And that’s just regarding helm and cloak graphical issues; the concerns are far greater when considering there are currently 12,988 objects, 11,037 NPCs, 47,451 item locations, 2,531 interface sprites...the list goes on.

---

Rest assured that the QA team work tirelessly on getting updates in the best shape possible for players, each and every week. Personally, being a large fan of MMO games (and typically being subscribed to many at a single time), you are getting every second worth of your time and membership investment into RuneScape.
Hopefully, this will answer some questions and raise more. Feel free to ask any questions you have on the forums. Happy adventuring!

sosolid2k

Aug 21 2009, 06:27 PM

*claps*

XD

Yeah, way too many kids seem to be playing. I've got a rant on forum atm about price manipulating clans

A few replies from the price manipulators themselves:

"lol i have 20k p pots... but they are not that high and u should just make ur own they r easy
and i am kinda srry merching ruines ur fun"

"I myself have 40k pryaer potions = (480mils worth)
cuase i was planing on just doing barrows forever, now i plan to sell in a couple updates
P.S.- I lover "Pyramid Scheme" Clans"

You can pretty much tell their ages from posts like that, they're just a bunch of selfish kids that don't give a shit about anyone else playing the game. All they're bothered about is earning as much money as they can, as easily as they can. It's people like this that Jagex are going to have to keep dealing with if they don't put a big effort in to get rid of them.

It's probably not Jagex's intent, like they said the game is aimed at college students and around that age group. But stuff like making the game easier and all the attempts to make things funny (like a lot of npc text, 'fun' items etc) just caters to the lower age groups and makes the game more appealing to them. They should really try to focus on the age group they are intending the game to be aimed towards, not just pleasing the current players. Older players tend to play longer anyway, most younger players will get bored a lot quicker >.>

The skill sounds exciting though c: Not really been that many 'good' updates this year so far, so hopefully they've got a few coming up including the new skill

The Rock 31

Aug 21 2009, 06:34 PM

what's the quick find code for your price manipulation rant? i'll support it because they are a big pain in the ass...