The Benefits of Contextual Inquiry

by Michelle Jewell

As our CEO, John Howard likes to say, “We’re one of the only agencies that eats our own dog food.” What he means with this old-school tech phrase is, as an agency we create SaaS products and we use these digital tools ourselves. The product this research was concerned with is called Sluice – a workflow management mobile application based on block-chain database. The product works by seamlessly integrating with a client’s backend system, enabling them to post, manage job assignments and collect relevant information in real-time.

Sluice can be adapted to any workflow project, currently it’s being utilized to take the property evaluation process mobile by enabling the gathering of all the information required for a property evaluation in the app.

The app has been tailored to incorporate useful technologies that allow the appraiser to get directions to the property, monitor their progress, generate floor plans, upload GPS tagged photos and fill out the required appraisal and inspection forms. Should the job be off the grid, Sluice is specially built to work just as well offline, uploading data to the servers once the connection is restored.

The target of this research and development spike was the introduction of a very specific form required by one of the largest mortgage banking institutions. We also used the opportunity to conduct research into utilizing ancillary work forces, already engaged in the property sales and management industry, and whether they could fill the manpower gap in a diminishing appraisal industry. This is motivated by the success of the gig-economy and the number of people embracing this as part of their standard income.

So, what do you do when the Sluice app is intended to address production shortages brought about by an aging workforce? What do you do when that same workforce is not particularly tech-savvy? Can a potentially inexperienced alternate work force, exposed to a heavy dose of unfamiliar industry jargon, do an appraisal correctly?

You do UX research! And you take that research into the field.

Background and goals of Contextual Inquiry

The goals of this testing were two-fold. First, we wanted to test a new form.

Sluice users visit different sites to make reports about features they observe in-person. However, the new form we were testing was significantly longer than anything else we had used in Sluice. As a comparison, a property and market condition report has one screen. The form we were testing requires users to fill out more than 100 screens of information! There are multiple ways to organize such a behemoth and designers and developers were going to be making a lot of decisions in a vacuum. That’s never good.

Second, we wanted to test the differences between the two groups of professionals.

The theory going into testing was that other real estate professionals who are already accustomed to visiting and evaluating single-family homes and insured to enter them might be a possible substitute – for having a 2nd workforce gather the data in the field for a centralized appraiser to review. In this study, we explored whether real estate agents could accomplish the work.

So, to test these two goals, we recruited an equal number of real estate agents and appraisers. Then we rented two houses (to check if the type of property made a difference. Spoiler: it didn’t.) Users showed up to the house one at a time. We opened the app to form, told users that they were appraising the house they were standing in, and set them loose. Go. Yours truly and one designer observed them as they filled out the form.

Getting out there makes a difference

All research produces more findings than you expect. If you train yourself to be open to them, they can become critical observations that drive design. In this case, those unexpected gems were all location-based. For example,

Excuse me, I need to step back outside. Doing the hokey-pokey in and out of a stranger’s home is awkward during the best weather. Imagine it’s raining. Imagine a backyard scattered with dog-poop landmines. Imagine that the ill-tempered dog who left the poop is not restrained. It’s almost impossible to design or develop for any of these scenarios until you have been there.

It was cold outside, and it got dark. We already knew that our floor plan builder needed refinement. Imagining users pulling off gloves to complete the task had not previously been a part of that equation. New considerations such as day/night lighting transitions were added to the requirements. It wasn’t something we planned to learn, but it still makes a difference.

It’s weird doing work onsite. That pertained to us as researchers. Note to self, when a home is brand new there’s no wifi. More importantly, that’s something we have to keep in mind for the users who will be completing these forms. How can we make their work simple and quick?

Real estate is horizontal. Prior to testing, the app only supported portrait photos. Ten users beating the side of the tester phone pushed landscape orientation higher in the development cycle. Some functions are so essential to the appraisal profession that they can’t wait for the next version.

Does the user’s profession make a difference?

The difference between real estate agents and home appraisers was pronounced. Real estate agents approach a property subjectively. They want to be able to paint a picture of the place for current or future residents, presumably because they are either selling it or considering it as a match for a client. Their work on-site at a property is usually performed along with other people–other realtors, clients, etc.–so their approach is interpersonal and transactional. In testing, they became impatient with the task of filling in the form, preferring to talk to the researcher explaining their thoughts and opinions of the property. A decidedly different approach to the home appraiser. By way of demonstration, none of the real estate agents completed the task, but that is not to say they couldn’t be educated further.

Appraisers, on the other hand, approach a property objectively. They need to complete a just-the-facts-ma’am report describing the home that culminates in a determination of value. Their work on site is performed alone or possibly with a trainee. They may interact with the resident or agent, but only to ask questions that will allow them to complete their report. They are independently responsible for their assessment and are accustomed to defending their conclusions. In testing, they set at the tasks outlined in the form immediately. Any difficulty with performing the task or questions directed at the researcher came because they were unfamiliar with filling in data on a phone or that the form seemed ‘wrong’ compared to what they were used to. Not all of the appraisers completely finished the form in the allotted time but as a group, they completed far more than the standard workforce did.

So much of the research process is necessarily sterile and controlled. And it should be, sometimes. In this case, getting out of the lab–getting outside period—allowed us to make critical discoveries. The key is having a strong plan and the strength to let the plan get messy.


Lindsey Harrison headshotMichelle Jewell is our Senior UX Researcher and began her career in academia, she was the Instructor of the Communication Department, where she taught Public Speaking, Human Communication, Argumentation, Communication Theory, Interpersonal Communication, and Small Group Communication. Michelle then joined James Madison University as the Communication Center Coordinator. She also taught communication courses in the School of Communication Studies where she was responsible for data gathering and reporting while conducting research on communication consulting. In 2013 the University of Texas at Austin was fortunate enough to add Michelle to their team as Director, School of Undergraduate studies. She managed the Sanger Learning Center, the University of Texas’ largest learning resource providing more than 100,000 hours of content tutoring and study skills support to a third of UT’s 50K+ student population.