Two things in particular which set our Dashboard effort apart from other business intelligence or executive dashboard tools are the way we engage our staff in the process, and the extended integration we’ve done with core museum systems.

In the past several weeks, we’ve spent a lot of time discussing the theory and the underpinning logic of why any museum would consider moving forward with a strategy like this.  If you’re just joining the discussion you can find the previous articles here:

1. Walking The Talk
2. Reasons for Transparency
3. Impact on Institutional Culture

This week we’ll take a detailed look at how the Indianapolis Museum of Art implemented these ideas in the IMA Dashboard.  We’ll examine how we structured the experience, what our goals were and what some of the results and unintended consequences turned out to be.

We’ve had some really great questions and points raised in the comment sections of the previous articles, and I’d really like to hear questions / thoughts from the peanut gallery as it were.  Don’t be shy!  I’ll do my best to respond to every thoughtful comment in turn.

Transparency in Practice: IMA’s Online Dashboard

As discussed in previous posts, one of the most important aspects for museums wishing to take steps to be more transparent about their successes and failures is a decision about the best way to share this information.  There are many ways museums might accomplish this. Ideally an organization’s information should be freely available to all interested parties with a very low barrier to access. Many museums have adopted the practice of making their annual reports and even tax returns available online for public access. It would be difficult to make the argument that these mechanisms provide “easy access” to this information since these documents are often lengthy, technical, and difficult to interpret.  The investment required by a member of the public must be high enough to overcome these barriers in order to develop an understanding of the museum’s performance.  While well intended, this method of presentation obfuscates the information which, if shared in a simpler more user-friendly model, might otherwise lead to valuable interactions and discussions with media, donors and the general public.

In the summer of 2007, the Indianapolis Museum of Art began to take steps to capitalize on an institution-wide effort and commitment to organizational Transparency. A team of web developers and graphics designers led by the Chief Information Officer set out to design a presentation of information and statistics about the museum which would enable an at-a-glance interaction as well support of deep-diving investigations into specific topics of interest. The team desired a system which was easy to digest and easy to navigate, and could support the wide array of information important to the mission of a diverse institution.  The project took inspiration from contemporary web design and interaction trends to create a site which would feel fresh, fun and visually engaging.  Feeling that many corporate dashboards were both intimidating and hard to understand, the team strove for a simplicity of presentation that could hook interested visitors into a deeper investigation and tracking of the museum’s performance over time. Finally, the institution needed an easy to use tool which could be integrated into the pre-existing workflows and job demands of many of the different staff around the museum.  The result was a tool called the IMA Dashboard, which was launched by the museum in September, 2007 and later released as open-source software for the benefit of the larger museum community.  Many museums and institutions have downloaded this software and expressed an interest in using it to fuel similar endeavors within their own organizations.

Figure 1 - The IMA Dashboard

The selection of an initial set of statistics for the Dashboard was undertaken by the senior management team of the museum and focused on metrics which had direct impact and bearing on the mission of the museum.  Nine statistics were selected initially covering areas such as, the acquisition of art, educational tours, membership, financial performance, horticultural activities (the IMA is an accredited horticultural institution), and attendance.  Statistics are arranged by topic or by department and visitors are invited to explore related bits of information.

The automated reporting of statistics to the Dashboard was a topic of much discussion early in the development of this tool.  At first glance, the technical slickness and wizardry of an automatically updated dashboard seems to hold much promise.  In reality, several factors existed that complicated this fact and have led the IMA to a slightly different approach.  Remember first, that a key desire of pursuing a policy of institutional Transparency is not only to share that information with our external constituents, but with our internal constituencies as well.  The automation of statistics is convenient in that it saves effort and attention on the part of the museum staff, but can frequently and easily result in neglect.  This result can be attributed more to human nature than to any lack of effort.  Parents of young children can attest that we are conditioned at an early age to tune out automated processes, or expected interruptions.  This is, in fact, the opposite of what we want to encourage in our staff.  Since these are “mission critical” statistics, we want staff members to attend to and understand them each and every month.  If a small amount of data entry causes staff members to pause and internalize this data at the same time that it is deriving a benefit for the public, so much the better.  Secondly, computers are notoriously bad at predicting exceptions to the rule.  Take attendance counting for example.  Institutions can make rules about the hours the museum is open, and about when they expect certain numbers of staff to report, but museum staff cannot tell the computer how to anticipate a weather related closing, or a special event on a day the museum is normally closed.  These cases all require human intervention.  Likewise, there are many types of statistics which are important to the operation of the museum, for which no automated system currently exists.  The IMA tracks the number of hours conservators spend treating works of art in our collection.  We think it is important, that despite all their other responsibilities, conservators should continue to treat the works in our collection regularly.  While many different types of time-tracking software exist, our conservators already have a system of tracking this information that works well for them.  So instead of spending time and money purchasing, integrating and training staff so that we can automate this system, we simply ask them to cut and paste their hours into a web form once a month.

The involvement of museum staff in the selection, authoring, and maintenance of these statistics is a fundamental priority for the Dashboard tool.  As discussed earlier, utilizing members of the staff in the collection, reporting, and monitoring of key statistics is an important way to engage them with how and why certain numbers are the way they are and how the museum might improve its performance over time. Staff members from various departments were invited to submit statistics which they felt would be important to track.  The web team worked with these staff members to author, refine, and create the statistics they suggested.  Each statistic is assigned to the museum staff member most directly responsible for monitoring that information.  Ideally, updating the Dashboard will coincide with existing workflows and reporting tasks that staff members are already performing. The frequency at which each statistic is updated varies depending on how quickly that information changes, from five minute updates for automated attendance counting to twice annually for horticultural plantings on the museum’s campus.  Email reminders are sent to the assigned staff members and updates consist of a simple cut-and-paste operation taking less than one minute and very little technical knowledge. As of this writing, the IMA Dashboard maintains statistics in 6 different topic areas across 13 departments.  42 statistics are current tracked and maintained by 31 staff members. These statistics have been maintained by the staff usefully for over two years as of this writing.

Several of the statistics recorded become quite interesting and even compelling examples of improvements and challenges encountered by our museum during the past year. For example, the museum has a commitment to reducing our energy consumption.  As a result, we track the total kilowatt-hours per month we consume as stated on our gas and electric utility bills. (See Figure 2) We have been relatively successful in this endeavor and have in fact reduced our energy consumption year-over-year in 30 of the last 34 months.  In February 2009, we used 22% less energy as compared to just one year earlier, while in September 2009 we used just 2% less energy than September of 2008.

Figure 2 – The tracking of energy consumption on the IMA Dashboard

Not all statistics are so rosy however.  Financial statistics show that, as a result of the economic downturn, the value of our endowment holdings decreased by almost $100M in a six-month period between September 2008 and April 2009.  Statistics show a 46% shortfall in contributed income compared to the budgeted projections in December 2008, and an adjustment of those revenue estimates in the following month. Early in 2009 the museum went through a strategic restructuring of staff positions resulting in the elimination of several part-time and full-time staff positions – a fact which is reflected clearly on the Dashboard update of our Employee Count and distribution on March 10, 2009. While these statistics are not among the shining examples of museum performance, we are thankful for them. They have given us a chance to engage with donors, board members, funding agencies, and our local city government about the reality of the challenges the museum is facing and our plans for addressing them.  The way these “bad” statistics would be received weighed heavily on the minds of the senior management team as we discussed this series of tough decisions. The ability of the Dashboard to serve as a positive seed for discussion is explained by Anderson, “The point of the Dashboard is both to crow when we see positive trends, of course, but also to show where we have room to improve, either through our actions or as a result of the generosity and support of others.“

Overall, the Dashboard has been received very positively by the IMA staff and senior management in particular.

“The dashboard has been a persuasive tool to use with the news media. Certain journalists who we work with on a regular basis know to check the dashboard for facts and figures, though they’ll still often verify the information with our public relations staff. I believe that the transparency that the dashboard allows permits us to have additional credibility with members of the media. They know that we’re not hiding information from them.”

-Katie Zarich, Director of Public Affairs, IMA

“In my opinion, the IMA’s dashboard has served as a visible reminder of our accountability to our supporters and the communities that we serve.  It is remarkable to me that it is not the data that has gotten the most attention, but the mere act of posting the information for public view.  By openly sharing the information, it has helped to build a sense of trust with our constituencies.  Mechanically speaking, the Dashboard is so user friendly that it enables those who own the data to update it using minimal time and effort.  In order to be a sustainable tool, I think it is important to have this ability to disperse responsibility for its maintenance and to minimize the additional time burden on staff to keep it current.  The one thing I don’t think we have quite figured out is how to incorporate the Dashboard as an information source or motivational tool for staff.  It is a challenge to balance what is interesting and useful to outside users with the needs and interests from an internal management perspective.”

Anne Munsch – Chief Finance Officer, IMA

More recently, the Dashboard has been used to illustrate museum visitor demographics accomplished through the collection of zip codes from visitors to the museum’s campus. This admissions data is then correlated to demographic data about race, age, income and educational attainment in the museum’s local community.  The information is presented in a simple map interface which allows the public to explore these demographics at their leisure. (See Figure 3)  Attendance tracking software utilizes heat-sensitive cameras to detect and automatically count the visitors to the museum.  This data is automatically integrated with the Dashboard tool and drives graphs and charts allowing year-to-year comparisons and attendance projections. (See Figure 4)

Figure 3 – A demographic comparison of museum admissions by zip code

Figure 4 – A comparative graph of month to month museum attendance

Next week we’ll conclude this series of posts with a set of suggestions for museums to use when staging their own efforts online with transparency.  I’d love to hear about ways in which these articles are being discussed in your museums or particular struggles or flaws in these arguments. -Rob

First, a quick overview of some technical bits. The user can select a range of dates from the map interface, and a new set of markers is rendered after an AJAX query to the backend PHP code, which requests the data from our database and aggregates it at the zip code and state level. The decision to do this two level aggregation was based on the performance hit that would be taken in trying to render on the order of 7000 icons for a year’s worth of data on a nationwide map. The MarkerManager class (which used to be part of the core Google Maps API) is used to display one set of icons when zoomed inside of what might be called “state level” and another set when zoomed out further. This adds a bit of extra complexity to our colormap choice.

There are a number of ways that this data could have been mapped into visual symbols. The size or shape of the glyphs could be related to the number of admissions, for example. The method that we are using maps the number of admissions to color (when this mapping is stored, it is called a colormap). The particular mapping that we use is important, because some mappings are better than others for certain tasks. For the markers on the map, we use a colormap that smoothly transitions from a blue at the low end to a red at the high end. The particular choice of blue to red leverages our cultural understanding of blue as “cold” and red as “hot” (note that sometimes colormap choice depends on cultural interpretation), a metaphor which works well as a representation of low vs. high admission rates. The combination of these two choices allows us to understand the general trends in the data without needing to refer back to the legend frequently. The drawback is that it is a bit more difficult to compare the value of two individual markers than it would be if we had picked a colormap of more distinct hues such as {red, orange, yellow, green, blue, purple}. We tried to make it easier for the viewer to make comparisons by limiting the number of colors in our legend.

The next important choice is how exactly to map the number of admissions to a particular color. To do this well, an understanding of the distribution of the data is necessary. Intuitively, we would expect that there will be high admissions numbers for the zip codes closest to the museum, and the trend will be toward increasingly lower admissions from more distant zip codes, and likely there will be many zip codes with very low, but non-zero admissions numbers. The best way to look at this objectively is with a histogram. I will take a number of approaches with the data from the year 2008, for which the highest raw admission count for a zip code was 12,375 (during the aggregation, 9 digit zip codes are merged into 5 digit zip codes, resulting in slightly higher numbers in some instances), out of 6806 zip codes.

This first histogram segments the zip codes into buckets in a linear fashion, with each bucket representing 10% of 12,375. As you can see, the vast majority of the zip codes end up in the first bucket. If we used 10 colors with this linear mapping, almost all of the glyphs would be blue. It would be great for identifying and comparing the few zip codes that have very high admission, but a lot of subtlety in the big picture would be lost.

This histogram also uses a linear segmentation, but with an upper limit (or clamp). Any zip code with 10 admissions or more are put into the red bucket. This would be fine for comparing admissions from distant zip codes, but we wouldn’t be able to distinguish visually between admissions from zip codes around Indianapolis.

This is where the beauty of logarithms comes to the rescue. Let’s use our example to walk through the process. First we determine the logarithm of our maximum number, 12375, in base 2 (working in base 2 is very efficient for computers). The result is 13.59. The highest bucket represents 12375 in the real world, and 13.59 in exponent-land. Each bucket represents a tenth of 13.57 in exponent-land, just as in our first histogram each bucket represented a tenth of 12375. We then simply derive the numbers that each bucket represents in the real world; 2 to the 1.359th power equals 3 (when represented as a whole number), for example. These whole numbers in the real world are then used to put the zip codes in the proper bins.

The result is that we have teased out more meaning to bestow upon our colors. There are still thousands of zip codes in those blue buckets, but they are the ones that are scattered across the nation on our map, so here we are aided by spatial distribution. In the red zone, there are dozens of zip codes in each bucket, and this is just what we want to be able to analyze the subtleties around Indianapolis where the icons are more spatially dense. It also turns out that this trend is pretty much independent of the date range, and the colormap still works out well at the state level.

On the actual map we use eight icons rather than ten to reduce clutter in the legend, but the algorithm is the same.

You might notice that we use a different type of colormap in the demographic windows. We actually tried using a smooth colormap for this data for the more linear demographics (Income, Age, etc.). The problem was that it was then very difficult to visually relate a particular shade of blue or green to a particular age or income level, which would almost certainly lead to eye strain and headaches. So here we used a more rainbow-like colormap to make that analysis easier.

In summary, colormap selection is a complex process involving aspects of mathematics, design, and the nature of the dataset. It’s one of those areas where art and science come together and the best practices of each can lead to successful communication of concepts and beautiful representations of data.

You can find a lot more graphs from the survey results in the rest of this post, but I thought it would be interested to just pull out a few interesting stats for you up front.

Stats Quick-Hit:

• 480 People took the survey between 12/22/2009 and 2/9/2009 (our web traffic during that time included 113,000 unique visitors and 450,000 pages served)
• Almost 90% of people who took the survey were satisfied or very satisfied with their experience on imamuseum.org
• The average visitor is Female (67%) and between 25-34 year old.
• More than 55% of website visitors use Facebook! Guess keeping the IMA’s facebook page flush with content is a good idea!
• Sadly 51% of survey-takers Never Comment on Blogs (c’mon… poke us a little, we don’t bite!)
• Survey-takers were overwhelmingly Caucasian. Our real attendance through the doors of the museum is different than this, so it seems like we need to do a better job of reaching some different audiences.

We’ll be contacting the lucky winner of the iPod touch this week and sending out the love to them!  Thanks to everyone for helping us with the survey!  We’ll be back in touch as we put together a new set of user personas to use for some upcoming web work.

In the mean time, enjoy sifting through our data for us.  If you see anything interesting here, be sure to be one of those 49% who do comment on blogs, and Let Us Know!  We’re happy to answer any questions you ask!

As numbers twirl around my head like 10-year old Chinese gymnasts (they are definitely NOT 16 that’s for sure), I thought I’d share with you some of my favorites. You can find a lot of these stats on the IMA’s dashboard, but some of the numbers are not accessible to the public…until now. In the Museum’s ongoing dedication to full transparency, here’s IMA by the numbers:

Full-time Employees: 197

Average Daily Blog Readers: 220

To Live Forever Exhibition Attendance through Aug. 11: 18,254

IMA Members: 10,454

Pieces of Glass in the Efroymson Entrance Pavilion: 225

Days until To Live Forever Closes: 24

Tickets Sold to Summer Nights Film Rocky Horror Picture Show: 714

Percentage of Web visitors from Indiana: 52%

IMA Curators: 12

Visitors to IMA, July 2008: 47,548

Cubic Yards of Concrete poured for 2005 renovation: 25,000

Square feet of IMA building: 492,000

Nugget Factory productions on YouTube: 100

Women named Pam on Staff: 5

Designers named Matt on Staff: 3 (2 graphic & 1 web)

Days until Power and Glory: Court Arts from the Ming Dynasty opens: 73

Number of Views to IMA’s first-ever YouTube video: 3,593

Number of days I’ve worked at the IMA: 1,354

]]> http://www.imamuseum.org/blog/2008/08/14/ima-by-the-numbers/feed/ 2 numbers