A scan is not a success: measuring an AR exhibition

A visitor scans the code, holds up a phone, lowers it and points out a detail to a friend. Another visitor scans three times because the page did not open properly. A dashboard might make the second encounter look busier.

That is the difficulty with measuring augmented reality in an exhibition: activity is easy to count, while a useful encounter needs interpretation.

An artist may want a visitor to notice how a drawing changes over time. A curator may want to explain a missing part of an object. A gallery may want people to feel comfortable asking about a work. Those are different ambitions. A convincing evaluation begins by naming one of them, before a number becomes the project's definition of success.

What should someone leave with?

Consider a hypothetical AR layer showing a painting's earlier composition. A weak objective is “increase engagement.” A clearer one is “help visitors recognise that the artist changed the position of the figure.” The second gives a team something specific to listen for and something concrete to improve.

The point is not to turn a gallery visit into an exam. A response might be a question, a gesture towards the canvas or a conversation between visitors. The team needs to decide what evidence would reasonably suggest the interpretation is doing its job.

This approach has a useful precedent in museum practice. Inclusive Digital Interactives: Best Practices + Research, produced through a collaboration involving Access Smithsonian, discusses formative evaluation during development, using observation and visitor responses to improve experiences. Evaluation is more helpful when it can still change the work.

For choosing that interpretive purpose, return to museum AR that serves the artwork. Measurement should follow the curatorial intention rather than quietly replace it.

A scan is an event, not a person

A QR scan, a page opening and an AR scene starting are different events. A visitor can produce several of them. Several visitors can also share a single phone. Staff testing may appear in the same totals unless it is identified separately.

Before putting a figure in a report, write down its meaning. Does “view” refer to the page, the camera experience or a piece of media? Does a repeat visit count again? Can you tell which events came from the exhibition and which followed a link shared elsewhere? If the system cannot answer a question, keep that limit attached to the result.

Be equally careful with time. A short session may mean the idea was communicated quickly. A long one may mean someone was absorbed, interrupted or struggling. Average duration alone cannot distinguish those situations.

Watch the small moments between the numbers

During a pilot, observe how people encounter the invitation. Do they recognise that the work has an additional layer? Do they understand whether sound is involved? Is there somewhere comfortable to stand? Does the camera request arrive before they understand why it is needed?

Then watch where attention goes. A person may spend time searching for the trigger image, inspecting the animation or comparing a revealed detail with the physical surface. Those actions can take a similar amount of time while meaning very different things.

A short, optional conversation can help. “What caught your attention?” leaves room for an unexpected answer. “Did our amazing AR make the artwork more engaging?” mostly reveals the answer the interviewer hopes to hear. Ask about the work as well as the technology.

Keep observation proportionate and explain any participation clearly. There is no need to collect identifying information simply to find out whether a label makes sense. Record practical barriers and recurring questions without turning individual visitors into profiles.

Include the people who could not take part

Feedback from people who completed the experience misses those who could not begin. A phone may be unavailable, the text difficult to read or the required movement uncomfortable. Someone may simply prefer to keep their phone away.

That does not make every non-participant a failed conversion. It does mean the exhibition should remain intelligible without compulsory participation, and the evaluation should include alternative ways of encountering the content. Our article on accessible AR exhibitions considers those choices in more detail.

Record the circumstances of the pilot too. Quiet mornings, guided groups and crowded openings produce different situations. If staff actively introduce the experience, say so. Their contribution is part of what you tested, not an incidental detail to remove from the report.

A comparison can help, with honest limits

Suppose visitors repeatedly miss a painted detail. The team tries a shorter invitation beside the same work and observes whether people find it more readily. That is a useful improvement question. It does not establish that all AR increases learning or that the same result will appear in another museum.

Where practical, compare similar conditions and change one meaningful element at a time. Note differences that you could not control, including attendance, group composition and staffing. Small pilots are good at revealing usability problems and promising directions. They are much less suited to sweeping causal claims.

The Chicago History Museum team describes this distinction in its account of evaluating AR and VR. Enthusiastic reactions did not establish what people were learning. In an evaluation of a Chicago00 VR experience, observation and interviews revealed that some participants' sensory absorption competed with attention to narration. That is a finding from a particular VR project, not a verdict on every immersive exhibition. It shows why the questions behind the numbers matter.

Count the work the institution is doing

A successful pilot also needs a manageable place in daily operations. Record how often staff help visitors, how long revisions take and whether anyone knows who should respond to a broken link. Include preparation, access alternatives and content review alongside software costs.

This is one of the questions behind a MuseumPros discussion asking whether museum AR is worth the effort. Contributors describe very different experiences. Their figures are unverified and cannot serve as benchmarks; the useful lesson is to examine usefulness and effort together.

For an artwork-led pilot in Painta, the sensible starting point is one focused experience and a question you can actually observe. Use whatever event data is available according to its documented meaning, then add direct observation. Do not assume a platform can measure comprehension.

The strongest report may be modest: visitors could start without help, several noticed the intended detail, and others needed a clearer invitation. That gives the artist and curator something to work with. A large scan total, standing alone, does not.