Our application contains diagrams, which are stored in a configuration file as embedded metafiles. Part of the application allows new diagrams to be added to the configuration file. A colleague commented recently that our application seemed to lock the .emf file containing a picture being added – once the picture was added, you couldn’t change the .emf until you closed our application down.

When I started investigating, I discovered we call GetEnhMetaFile( filename ) when loading the metafile to add to the configuration file.  We call GetEnhMetaFileBits to copy the binary data for saving into the config file.  And we call SetEnhMetaFileBits when loading the binary data back from the config file.

And that’s it. No clean-up, ever.

So no wonder the file is still locked. (Unfortunately, although our automated tests spot memory leaks, they don’t spot resource leaks yet.  Any suggestions?)

After poking around a bit (the engineer who wrote that section is no longer with us), I discovered it’s not clear when it is safe to delete the metafile, because metafile handles are shared around so that there’s no way of knowing whether something else is referring to the same metafile, or even where it came from in the first place.

So the simplest way to deal with this is to take a copy of the metafile when assigning it to a diagram and then close the source handle at the place itwas opened.  That way the lifetime and ownership is clear.  The loading routine is responsible for its copy, and the diagram is responsible for its copy.  The delete is clearly near the allocation.

How do you copy a metafile?  Hmmm.  There’s no CopyEnhMetaFile function.  We’ll need to get the bits from the source and set them to the destination.

GetEnhMetaFileBits takes a handle, a buffer and a buffer length and fills in the buffer from the metafile.  Great…but how big should the buffer be?  Simple – you call it first with the buffer set to NULL and it returns the size you need.  Allocate a buffer of the right size, call it again, and Bob’s your uncle.  You’ve now got the binary data in a buffer which is in the perfect format for calling SetEnhMetaFileBits, and you know exactly how long it should be.

// CopyMetaFile version 1 - don't use this one!
WM_RESULT CopyMetaFile( HENHMETAFILE picture, HENHMETAFILE *phCopy )
{
  WM_RESULT       result;
  HENHMETAFILE    hMetaFile = NULL;
  UINT            buflen = 0;
  LPBYTE          buffer = NULL;
  UINT            retval;

  buflen = GetEnhMetaFileBits( picture, 0, NULL );
  if ( !buflen )
  {
    result = WMR_BAD_FILE_FORMAT;
    goto error;
  }
  buffer = (LPBYTE) malloc( buflen );
  if ( !buffer )
  {
    result = WMR_RESOURCE_FAIL;
    goto error;
  }
  retval = GetEnhMetaFileBits( picture, buflen, buffer );
  if ( !retval )
  {
    result = WMR_BAD_FILE_FORMAT;
    goto error;
  }
  hMetaFile = SetEnhMetaFileBits( buflen, buffer );
  if ( !hMetaFile )
  {
    result = WMR_BAD_FILE_FORMAT;
    goto error;
  }

  *phCopy = hMetaFile;
  return WMR_SUCCESS;

error:
  return result;
}

But what about the buffer we’ve just allocated.  Who owns that?  Is it still needed by the metafile, or has the data been copied out of it? Testing with this definitely shows a memory leak, but maybe the metafile still refers to it and freeing the memory again would be dangerous.

MSDN was silent on the topic, a quick web search found nothing either.   The only option was to try it and see.  So free the buffer again after calling SetEnhMetaFileBits.  Quick compile, run the program – it all seems to work.  We still get the pictures, and there’s no crash.

But maybe we got lucky – maybe the memory hasn’t yet been reused, and the program is just accessing the stale, deleted, but not yet cleaned up memory, and it will crash in half a hour when something else gets given that memory during an allocation.  That would be a horrible bug to track down.  How can we make certain?

I know – let’s corrupt the memory immediately before we delete it.  A simple memset to 0 should be enough to wipe out any metafile information, and that would be instantly visible in the application as all the metafiles would just be blank.  Compile, run…no everything still works perfectly.

Okay, maybe I’ve not managed to corrupt the memory properly. (You may think I’m being overly cautious here, but with asynchronous programming, paranoia is definitely your friend.)  Maybe Windows somehow manages to cope, while still needing the memory there.  Maybe the build didn’t pick up the change properly. How can I check that?

Well, what happens if I corrupt the memory before calling SetEnhMetaFileBits?  Let’s move the memset to before the call.  This time no pictures, and in fact the app crashes.  So we’re definitely making a difference.

Move the memset back after the SetEnhMetaFileBits and try again.  Does this work again?  If so, we’re on a winner.  And indeed everything works perfectly again.  So SetEnhMetaFileBits does take a copy of the memory, and we can free our temporary buffer straight away.

So the final routine is as follows:

////////////////////////////////////////////////////////////////////////////
///
//  Function: CopyMetaFile
///
/// @brief  Creates a copy of the metafile.
///
/// Written by Ian Brockbank.  Provided with no warranty whatsoever.
/// Feel free to use for any purpose you see fit.
///
/// @param  picture     Metafile to copy.
///
/// @retval WMR_SUCCESS             Succeeded.
/// @retval WMR_BAD_FILE_FORMAT     The metafile couldn't be parsed.
/// @retval WMR_RESOURCE_FAIL       Couldn't allocate a new buffer.
///
////////////////////////////////////////////////////////////////////////////
WM_RESULT CopyMetaFile( HENHMETAFILE picture, HENHMETAFILE *phCopy )
{
  WM_RESULT       result;
  HENHMETAFILE    hMetaFile = NULL;
  UINT            buflen = 0;
  LPBYTE          buffer = NULL;
  UINT            retval;

  buflen = GetEnhMetaFileBits( picture, 0, NULL );
  if ( !buflen )
  {
    result = WMR_BAD_FILE_FORMAT;
    goto error;
  }
  buffer = (LPBYTE) malloc( buflen );
  if ( !buffer )
  {
    result = WMR_RESOURCE_FAIL;
    goto error;
  }
  retval = GetEnhMetaFileBits( picture, buflen, buffer );
  if ( !retval )
  {
    result = WMR_BAD_FILE_FORMAT;
    goto error;
  }
  hMetaFile = SetEnhMetaFileBits( buflen, buffer );
  if ( !hMetaFile )
  {
    result = WMR_BAD_FILE_FORMAT;
    goto error;
  }

  //
  // SetEnhMetaFileBits has taken a copy of the buffer, so
  // release our local copy.
  //
  free( buffer );
  buffer = NULL;

  *phCopy = hMetaFile;
  return WMR_SUCCESS;

error:
  // Clean up.
  if ( hMetaFile )
  {
    DeleteEnhMetaFile( hMetaFile );
    hMetaFile = NULL;
  }
  if ( buffer )
  {
    free( buffer );
    buffer = NULL;
  }

  return result;
}

So there you go.  SetEnhMetaFileBits takes a copy of the data buffer passed in, and the caller can release the buffer immediately after the call.

This code comes with no warranties, but feel free to copy and reuse it as you see fit. It worked for me, but I make no guarantees it will also work for you, and maybe there are other holes I didn’t think of which will catch me in future. If you do find a problem with the code (other than stylistic), please let me know and I’ll gladly try to fix it.  Otherwise, enjoy!

“It takes elite performers a minimum of 10,000 hours of deliberate, focussed practice to become experts.  That means it takes about ten years just to arrive at the top level.  And world-class performers don’t stop practicing once the reach the top; they keep up their deliberate practice, continually putting effort into getting better than others in their field.”

Ten years.  Of deliberate, focussed practice.  At least.

Raw talent helps, of course.  But it only gets you to the starting line. “After that, the people who excel are the ones who work the hardest.”

I’d better get back to work then…

When I’m looking at a CV, I want to know about what you have developed.  What meaty projects have you done?  Can you even program at all?  I want to know you’ve done more than learn some facts to pass some exams.  If you’re going to be working for me, I don’t want to be teaching you how to write code.  I want you to come on board and start contributing immediately.  So you have to have solid programming experience already.

Now, I know universities give you exercises and projects which give you plenty of opportunity to practice (at least, the universities I’ve been involved with do).  So tell me about them.  Sell yourself.  Show off the tricky problems you’ve solved, the slick interfaces you’ve created.  Well-rounded individuals may be interesting to the HR department,  but if you can’t program, I’m not interested.

If you do get through to an interview, I’ll be looking for some project in your CV to ask you about in great detail to satisfy myself that 1) you know what you’re talking about, 2) you did actually develop it yourself and 3) you can think on your feet and extrapolate.  If I can’t find anything to ask about, I’ll not bother inviting you to interview.

You’ve only got a minute while I scan the CV.  Don’t waste it.  You may know you’re the best programmer on the planet, but you need to prove it to me, too.

Resiliency is the ability to respond to an unexpected occurrence, deal with it and return to a state close to where you were before.  Remember the example of Fedex, who have a certain number of planes only 60% full, thereby having the ability to flex for extra loads and get back close to where they were.

An example given of an organisation becoming more resilient described how they network to build up links between departments and different areas. By getting to know each other better, when a problem arises, they have several benefits.

• They know who they can call on for help, and it’s not such a big hurdle calling someone they already know.
• They know the skillsets of  their colleagues, and hence who is most likely to be able to help with a given problem.
• People who are being asked for help know and trust the person asking better, so are more likely to help.
• People know better what backup they have to call on, so can factor this into their plans in advance.
• When the unexpected occurs, more people are involved, so more people have the chance to learn from it.

Again, an agile principle turns out to build better reliability: “We value Individuals and Interactions over Processes and Tools“.

Wolfson Microelectronics is seeking enthusiastic, degree qualified engineers with 3+ years experience in graphical software application (GUI) development to join the applications software tools team. The role is to provide software tools that guide the customers through the configuration of our devices to allow them to evaluate the feature set and performance and optimise the configuration of their products.

The position requires an ability to communicate clearly with customers and the design team and develop an understanding of customer tool requirements. This information must them be used to produce high-quality, intuitive tools. The candidate will also be expected to feed back useful information gained from the customers to the rest of the applications software team, design team and marketing team to help improve our products.

Travel to customer sites in Asia, Europe and the US will be required.

Ideally the candidate will have an excellent knowledge and developed expertise in the following :

• Excellent programming ability, ideally in C/C++ although related languages (particularly C#) may be acceptable
• Demonstrated experience in user interface development, ideally for Windows operating systems
• Strong diagnostic and analytical skills, with excellent attention to detail
• An innovative, creative, lateral-thinking problem solver
• Able to work to tight and variable timescales
• Good customer facing skills

Additional skills that would be highly desirable would be :

• Usability testing
• Asynchronous programming
• Microsoft Visual Studio, MFC, XML
• C#/.Net, WPF, Managed/unmanaged code interoperability
• Digital signal processing, Matlab
• Agile software development
• Embedded/firmware programming
• Experience of supporting a world-wide customer base

If interested, please send CV plus cover letter to cv@wolfsonmicro.com

Last week I was advised to hire an external usability expert for a couple of months.  Now, bruised ego aside, there are several reasons why this is a waste of time and money.

We sell mixed-signal chips – audio, scanner, power management, microphone, touchscreen.  I am responsible for the software which customers use to evaluate our chips and configure them to get the best out of them.  The first version was released about three years ago, after about a year of development.  There were some minor niggles with the interface, but it was good enough to release and get customer feedback.

Almost immediately, the team (that’s me) was diverted onto customer support for the best part of a year.  Since then I have been under pressure to add support for new chip features so that customers can use them and we can actually sell the chips.  There has been a general move to push the grunt-work of calculating coefficients up into the software so that what you actually programme the chip with are a set of magic numbers.  The only way to calculate some of them is actually to simulate part of the chip, which isn’t something we’re going to tell the customers how to do, so the bottom line is that my software has to do these calculations or you can’t use those features.  Those features were key to securing some big design wins last year.

We have also had feedback on the basic interface.  Fundamentally it is sound, but it has some scalability issues with the more complex chips we’ve developed over the last couple of years, and there are a few niggles which range from quirky to annoying.

So I had the following choices:

1. Provide support for the filters. Business case: enables millions of pounds in revenue.
2. Clean up the basic interface so that existing users have a slightly better experience.  Business case: er… some users (mainly internal users) have complained a bit.  Improves the perception of the company slightly.

What would you do?

I chose 1.  It has now been suggested we get a usability expert in because I haven’t addressed 2.

Maybe it would be useful to get a usability expert in.  However, a good start would be to get someone allocated to work on it.  The fact that I haven’t done anything about it doesn’t mean I’m unaware of the problem, nor does it mean I have no ideas how to improve matters.  It’s a simple issue of time and priority.  And however many experts you bring in, they won’t solve that.

By the way – if you have UI skills, you could be the person to work on this. We’re hiring.

It talks about how high-reliability operations (HROs) such as Accident and Emergency centres, Aircraft Carriers, Nuclear Power Stations ensure they have fewer than their share of “brutal audits” (aka disasters) despite having a very high potential for them.

The key attributes they identify of such high reliability organisations are:

1. Track small failures
2. Resist oversimplification
3. Remain sensitive to operations
4. Maintain capabilities for resilience
5. Take advantage of shifting locations of expertise

Reading this, I have been struck by parallels with the Lean Principles (in particular as presented by Mary and Tom Poppendieck).

Track small failures = stop the line mentality.  Don’t let errors build up, but spot them early and fix them early.  If something unexpected happens, stop there until you understand it and know how to handle it.

Remain sensitive to operations = push responsibility down.  Make sure the people with the knowledge to understand the situation are empowered to deal with (and/or escalate) problems – but make sure they have backup they can call on when the situation moves beyond their area of expertise.  An example was cleaners at a nuclear power station were changing air filters every couple of days because of rust.  After about six months, it was discovered a section of the 6 1/2 inch-thick containment chamber had corroded to the width of a pencil eraser.  If someone had thought that changing the filters every couple of days wasn’t right, this might have been spotted before it got so perilously close to a disaster.

Maintain capabilities for resilience = decide as late as possible.  The example given here is Fedex.  Every evening 20 to 25 of their fleet of airplanes start their journey to the Memphis hub only 60% full.  This means there is spare capacity if they get a late addition, which can be diverted if necessary.  To control this, they say the plane MUST arrive at the hub by 1:30am.  If extra time for the diversion plus half an hour on the ground for loading still allows the 1:30 arrival, it is authorised.  Otherwise the extra load is delayed – don’t jeopardise the whole flight on the chance of making up ground.

You can find Managing the Unexpected on Amazon.

So it is with software development.  When all’s rosy, anything will work.  But when the pressure’s on, what you do can make the difference between delivery and a death spiral.

Well – it’s an analogy which serves…

This is a blog about software development, with musings related to Agile development, asynchronous programming, and students.

Why have I started this blog?  I’m a software engineer based in Edinburgh, Scotland. I am continually learning about tools and techniques, and I am often surprised to find out that other people haven’t heard of techniques I rely on.  I’m particularly interested in Agile and Lean development techniques, and processes which help maintain quality (or at least highlight issues early).

I also grew up with asynchronous, multi-threaded, multi-processor, multi-computer programming when I started my career at DIGITAL; since leaving I have been first astounded, then amazed and now just resigned at how poorly asynchronous programming is understood.

And finally, I am involved in recruitment, particularly of students.  If I can just help some people develop their skills in the right way and sell their abilities rather than their buzzwords, I will have served some use.  I have seen a lot of CVs and a fair number of chancers whose claims don’t stand up to interrogation.  I will post thoughts on this here as well.

I don’t know if anyone will find my musings useful, but I hope they help someone somewhere.