NHC 2010 – Warts and All! 17

As the organiser for this year’s SOBA National Homebrewing Competition, I’ve now seen every aspect of NZ’s premier homebrewing competition from the other side. I have to say, I have a newfound respect for all the previous years judges, stewards, and organisers. Something that’s always intrigued me, as an entrant, is exactly what goes on from when you submit your beers, through to them being judged and presented. I thought I’d write a bit of a novel to let you know what we do, how and why we do it, and what I think we can do better next year. I’m not going to pull any punches on this, in the interests of full and open discourse, and it’s going to be a LONG post…

First, some background. I took up the challenge of running this year’s NHC because the previous organisers felt they’d come to the limits of the time they could donate. I understand why. This is NOT an easy job! I didn’t want to see the competition fall on the floor, and no other offers to organise it were forthcoming, so I thought “how hard can it be?” and took up the challenge. Well… it was a lot harder than I thought.

First, we needed a good website to act as the focal point for rules, style guidelines, answering questions, and taking entries and payments. Brendon MacKenzie made this side of it a dream by modifying last year’s website to do the job. Something I learned though is that people do not read. Something can be specifically stated on the website, and still entrants will ignore it. The prime example of this was brewers notes. The FAQ says when and why to enter a note, and advises that in most cases the brewer shouldn’t need to. Despite this, far too many entrants put things we simply didn’t need to know in the notes area – such as “all grain”, “5.2 abv”, “34 IBU”, “finest quality malt”, etc. Notes are for advising the judges of the brewer’s intent IF the beer deviates from style, or if it is in a specialty or “open” style. In this case, notes are an absolute MUST, and too many entrants didn’t provide any. I hadn’t realised how frustrating this could be until I took my turn at the judging table, but more on that later. Suffice it to say that several beers missed out on better scores, and even medals, due to lack of correct notes.

Something I didn’t take into account is just how much room nearly 600 beers can take up. I had planned to store the entries in a portable chilling trailer, but this fell through, so I prevailed on a friend (and fellow trainee judge, Phil Murray) to lend me a cool interior room at his workplace to store all the beers. Because it was a workplace, we only had access during working hours, and because I was judging this year, I couldn’t see the entries. Thanks to my wife Alexandra, and Phil’s fiancΓ©e Beth, for all the work they did in cataloguing and sorting/boxing up the entries. This job took two people the best part of 30 hours between them over two weeks. Entrants next year, please remember that checking every individual entry and letting you know if it has arrived is a little frustrating. Just trust us that we will let you know if it doesn’t show up, or if it was damaged. Thankfully this year, there were very few casualties of the postal system. Commiserations to those two brewers who did have problems.

In terms of a venue to hold the event, we couldn’t have been luckier. John Edmonds, club manager for the Ruakura Campus Club, offered us the RCC at just the cost to pay the cleaner. This gave us an excellent venue with a kitchen, chiller, and glasswasher (absolutely essential) allowing the judges to focus on judging and the stewards on getting the beers in front of the judges in excellent condition.

Sponsors have been great to us this year, with Farra Engineering offering a 30L stainless fermenter to the champion brewer, Hallertau offering to brew the “best in show” beer (and Steve’s going to have fun with this one), and Liberty Brewing offering “win your beer back” kits for the best ale and best lager.

Happily, the judging weekend itself went extremely smoothly, though we had one judge pull out a few days beforehand. Luckily for us, we roped in ex-winemaker, sometime brewer, and all-round great guy Simon Henderson to fill the gap. He did an admirable job at short notice, and really helped us out.

As for the judging itself, this is an area I have a lot to comment on. My goal, communicated to the judges right from day one, was to provide the best quality feedback ever from a NHC. To this end, we recruited some of the best judges in NZ. Our panel consisted of two World Beer Cup judges (Graeme Mahy and Geoff Griggs), backed up by a solid team of very experienced judges, and three “trainees”, Allan Hawkes, Phil Murray, and myself. In order to deal with the 294 entries, we ran three tables, each with two experienced judges and a trainee. After much deliberation, I made the call as organiser that only one scoresheet per beer would be completed. This forced the judges to collaborate, and discuss their impressions at length for each beer, before those thoughts were transcribed to the scoresheet by the table scribe (being the trainee judges – I apologise for the spidery scrawl from table 3, that was me). I believe this achieved the goal well, though there were still issues…

The main problem was the scoring itself. One of our judges was quite vocal in his opposition to the BJCP scoring system, and half way through the first day, I was fully in agreement with him. We thought that it meant the judges spent far too long thinking about the numbers, and not enough time analysing the qualities of the beer in front of them. The three different tables tackled “the numbers” in different ways. For example, table three (myself, Kieran Haslett-Moore, and Geoff Griggs) decided on medal worthiness first, then worked out what the scores should be to achieve that result, while remaining within the realms of reality. Table two (Graeme Mahy, Albrecht van Wallmoden, and Phil Murray) worked through the individual scores for each area, then adjusted up or down to fit their idea of medal-worthiness, and table one (Ben Middlemiss, Simon Henderson, and Allan Hawkes) just seemed to give a lot of zeroes! OK, seriously, they were harsh, but they were fair. They discussed the beer in depth, then discussed the scores. This led to a very well thought out score, though I often wondered if it caused them to over think it and talk the beer down to a lower score. For interest’s sake, at the end of day one, table three had the highest average score, and table one the lowest, though there was no more than a point in it, so judging methods resulted in similar scores. For improvement, I’d spend some time next year ensuring all judges agreed on the same method of evaluation and scoring, and perhaps institute cross table sampling to ensure calibration is maintained. I’ll come back to the scores later…

Timing was difficult. Some beers required a lot of discussion. Some were horribly infected and really couldn’t be judged beyond “infected, fix your sanitation”. For the record, table three scored these beers “12” and gave few comments. The other tables gave zeros. What can I say, we’re nicer. πŸ™‚ Anyway, because some beers took a long time, and some were quick. This made planning flights and breaks tricky, and there was a fair bit of stress involved knowing we had to get through nearly 100 beers per table over one and a half days. We did manage it though! Ten minutes per beer seemed like a fair average, allowing for much discussion and good feedback to be given. I apologise to any entrant who feels they didn’t get detailed feedback. We really did do our best, but I’m sure a few slipped through with a little less detail than we could have given. Some beers (usually the very good, and the very bad) were hard to comment in detail beyond “well done”, or “you really need to fix your sanitation/fermentation/yeast health etc”.

Kieran, who judged at last year’s competition, confirmed my suspicion that this year’s standard was a lot lower than previous years. There were far too many badly infected beers, and so many so far out of style that the judges were just left scratching their heads and wondering “what the hell was the brewer thinking with this”? Brewers, be very careful when you enter your beers. Choose the style well, and always taste it with the style description in front of you, and a brutally honest friend or enemy to confirm or challenge your selection.

Finally, a note for entrants on bottle variation. While we were cleaning up, the stewards, Phil, and myself decided to play “beer roulette” with the remaining entries. We’d pick a bottle at random, find the judging sheet, and see how it compared with the judging sheet. Most of the time, it was spot on, fully justifying the judges opinion. Other times it was MILES out. As in, there would be no off flavours mentioned on the sheet, but the beer in the glass was a train wreck of diacetyl or other faults. Or vice versa. The only explanation for this, given how accurate the judging seemed to have been, was variation between entered bottles. Having experienced this with my own brews, I’d suggest several brewers cost themselves medals by having a good bottle and a bad bottle, and the judges got the bad ‘un. Also, if you know your bottle foams, PLEASE advise this. Many stewards got slightly damp, and many beers presented poorly due to the foaming stirring up the yeast.

I hope nobody was offended by anything in this post. The goal is so you can all see “under the kimono” of the NHC, and help make following years better for entrants, judges, stewards, and organisers. I hope it was interesting.

A reminder that results will be posted out by the 13th of November, and there is a presentation ceremony on the 13th at Hallertau Brewbar. This event is limited in seats, so please advise nhcawards@soba.org.nz to book a seat.

17 thoughts on “NHC 2010 – Warts and All!

  • Stu

    Cool, so I get respect for handling all incoming entries in year one (160) by myself? I remember it being a damn hard job. Thankfully Brendon helped out on the actual weekend as a super steward.

    That year we had one table of judges, who did about 80 beers each day… they also only looked at each beer once and once only. I really believe this is the only way to go if you are judging against style guidelines rather than other beers (and awarding medals for every medal worth beer). I wish BrewNZ would do this also but that is another blog!). All you really want to know is whether it is worth gold, silver, bronze or no medal. If you are looking at awarding best in class or best in show, you may well need some sort of scoring system to differentiate the beers that would come back for “play offs” (not necessarily BJCP – it may be as simple as Gold+, Gold, Gold- etc).

    Can’t wait to hear about some of the stunning beers.

    Well done y’all. I definitely know my way around the back office of a beer competition and it is damn hard work. Congrats on getting it all complete.

  • Hamish

    Thanks for all your hard work in organising this years NHC Greig!! A big thanks to all the judges and everyone else who made this years comp possible. Now I can’t wait for the results…

  • Kieran Haslett-Moore

    Cheers Greig. It is alot of hard work and we should all be very thankfull to you and Alex. The weekend went increadibly smoothly, far more orderly than many a pro comp.

    It was a shame that the standard seemed to go down this year but there were still alot of great beers. Over the last 2 years the bar has continued to rise but I think with amateur comps like this there is always going to be the point when it relapses somewhat as new brewers come on board and the odd experianced brewer goes pro.

    There was alot of phenolic infections this year, where as last year we were seeing the problems we get more commonly at pro comps, V.D.K (just cos I can’t spell the D word) D.M.S. and oxidation. On the plus we didnt seem to get to many acetic issues which is always the worst.

    And thankfully there were a whole bunch of great beers and a handfull of absolute stunners!

  • Martin Craig

    Well done Greig, and thanks.

    I was a judge on the first three years’ competitions. To put your comments into context, last year’s standard was especially high. 2008’s offering ahowed a lot of infected beer, more than 2007, but this may have been exacerbated by storage.

    Some years we tasted as a single panel and ended up doing 100 in a day, which was unfiar on the entrants and probably on judges too. The judging panel system not only spreads the load, but allows confirmation/benchmarking between the groups.

    Of course, this needs more judges, and I am sure the standard of judging has improved since 2007, especially drawing on the education and experience of the BrewNZ competition. The more judges the better, so get involved if you get a chance.


  • Simon

    Nice one Greig and thanks for organising. Looking forward to the feedback and don’t I hope that all mine went via table 3.



  • James P

    Awesome write up Greig, and obviously a big ask for the time the judges had.

    Whilst I accept there is a strict timeframe and judges decision is final, I am a little confused as to the statement about bottle variation. I thought we presented two bottles minimum to allow for variation as I would have thought it fairer on brewers to check the first bottle, and if an infection etc is present, then the second bottle be checked to provide a definitive result. If both bottle present the same, the the results are evident, else the “best bottle” presented would be a fairer representation of the brewers ability to brew a beer to style?

    Having had experience of one bottle in a batch being “off” and would be guttered to be judged on a rogue bottle.

  • greig Post author

    Hey James,

    I take your points, and I understand that it would be frustrating from the brewer’s point of view to be judged poorly on a rogue bottle, but:

    1. It is the brewer’s responsibility to ensure the bottles are consistent. That’s part of the entry, packaging your product so it presents its best before the judges.

    2. If we asked for a repour of every beer we thought might have been infected or suffering in some way, there’d have been no time to judge the beers. As it was, I believe there were no more than six repours over both days.

    3. As I said in the post, random sampling after the event showed that things were pretty damn good, most of the time, with respect to consistency. There were just a few that were clearly different to the beer evaluated by the judges.

    The second bottles are generally reserved for accidents (opening too vigourously causing foaming, which lifts yeast, or neck breakage causing glass to contaminate the beer), or “taste off”/second opinions. In 95% of cases, it will not be touched, until the thirsty and hardworking stewards attack at the end! πŸ˜‰ It is entirely up to the brewer to ensure their beer presents well in both bottles entered. It’s part of brewing, and therefore, part of what you will be judged on. Just as if you were a commercial brewer with a beer which presented poorly in some bottles, you wouldn’t expect the consumer (judge) to have to say “oh, maybe I’ll buy another bottle to see if it’s just that one” even though often, that’s what we beer geeks do! πŸ˜‰

    I hope that clears things up.

  • Graeme

    Thanks Greig, Alex, Martin and the team for all the organising – it matched the WBC in ‘precision’! And having no beer tipped over you when judging is always a bonus.

    To back-up what Kieran was saying, there was too many phenolic ‘infections’, something that shouldnt happen at a homebrew scale (well actually shouldnt happen at any scale) as you have more control over sanitation. My advice is clean, clean again and if youre not too sure clean it once more! However I think a lot of the phenolics was due to yeast ‘strain’ – I suspect too high pitching temperature followed by too high fermentation temperature. No matter what the yeast ‘specs’ are I would suggest a pitch and ferment at 20c max – OK there are variances to yeast strains but concentrate on getting your ferments right first!

    James; your comment re: bottle variation leads me to one comment – there shouldnt be any at a homebrew scale πŸ™‚

  • MrC

    Nice write up Greig, it’s nice to hear about what goes on behind the scenes. I appreciate the openness.

    I would be interested to know about the order in which styles are judged. Is there a standard order for beer competitions or do the judges decide based on the beers entered? Also, do the specialty beers with a base beer style specified get judged with the other beers of that style or are all specialty beers judges together?

  • greig Post author

    I’m all about the open, MrC! πŸ˜‰

    The head steward designed the running order. In general, we split styles over the three tables, with large numbers of entries in any given style being shared between tables. The order was generally in “palate impact” order, though with plenty of bread and water on hand, this wasn’t a major issue.

    The specialty styles were done as a group, though I’d consider changing that if we did it again. Having a “mini IPA” after a manuka smoked chipotle chilli beer seemed like a bad idea. Luckily, Kieran spotted this in the running order and requested a change. It was easily accommodated, and no palates were destroyed in the judging of said beers. πŸ™‚

    Now, speaking of “open”, have you opened my Punkin Image Ltd ale yet? πŸ˜‰

  • Rob Owen

    Well done Greig, thanks for taking the reins this year. It’s a bloody big job but I thinks it’s a great competition and growing every year. So you’ll have everything under control for next years event πŸ˜‰

  • MrC

    Haven’t tried you punkin ale yet Greig. Still waiting for the right moment. I’ll definitely let you know what I think. Cheers.

  • Richard Pettinger

    Thanks Greig, and all the team for all the hard work organising… My wife Tracy and I did the cataloging of beer and wine entries for a South Island (Amateur federation) competition a few years back, so we know a bit about the work required, but I’m sure it was not of the scale you just managed!

    Did those whose entries didn’t arrive OK get notified? I always wonder if I’ve packed mine OK. After all the care in the brewing it would be a shame if they didn’t survive the long journey from Dunnos. Also, while I’m sure my beers weren’t up to a high standard, it’d be nice to know they were still intact so someone had a good enough time sampling them!

    I enjoyed your story, and the comments from others.

    Thanks again.

  • greig Post author

    Hey Richard,

    Yes, all arriving late, not at all, or damaged (thankfully, none made it to us damaged) were notified. The standard of packing was excellent this year. I guess it needs to be, as it seems NZ post is hell bent on playing football with many of them. I wish someone would hurry up and invent a better system of cheap goods distribution that doesn’t rely on total cowboys trashing the packages in their “care”.

Comments are closed.