Ready to test out new techniques

When I first transitioned to this department in March 2010, I  attended two workshops in Ann Arbor at Merit Network. One was about Information Architecture and another was about Site Search Analytics. I learned a lot in these two days and I’m anxious to test out some of the techniques I’ve learned!

Here are some things I wanted to share based on what I learned:

Information Architecture

IA is all about helping people find what they need. Using a combination of statistics and field research will help you come up with different categories of content. But before you figure out how you are going to present this content, you need to figure out what you are presenting. Ask questions about an organizations goal, why they want a Website and what they want people to find there. Talk to the people who answer questions (e.g., switchboard secretary or helpdesk operator) about the organization.

Card Sorting

Use index cards to represent the content pieces. Lay them out to see what kind of pattern forms and what content relates to other content. Categorize things and see if there is commonality between items. This will help you form a good outline to start with.

Many organizations don’t choose to make connections between related content. So they set up with a site and as time goes by they don’t know where to add things and things end up in maybe not the best place.

When you feel you have a decent navigation together, try this navigation stress test.  It makes you look at your content differently and helps you spot anything you could have missed during your initial analysis.

Site Search Analytics

This was my favorite portion of the workshop — insight into human behavior. First we analyzed logs and tried to determine a user’s session and what they were looking for on a specific site. Did things change from line to line in the context of a session? Was the person at the library or were they at home?

What people are searching for on a site is what they want. So you can get an idea of what people are looking for in your site, regardless of whether it’s obvious in the navigation or not. You may need to roll your logs into spreadsheets in order to get value from it.

If you keep looking at your data you will come up with question after question. And this is good since it’s questions about your own data. Specific search terms will occur at certain times of the year sometimes. You will be able to spot this pattern and be prepared in the future.

Here are a few techniques to try:

  • Review top ten and top 20 search term results every week and chart it out over a period of weeks.
  • Review percentage of search exits — number of people who left the site after a search. Is it a specific term? Search this term and see what might be holding people up. Were they looking for something else perhaps?
  • Review low search terms. It may not indicate a problem but instead shows that people are finding what they are looking for.
  • Analyze queries that returned zero search results. Why are people looking on your site for this content and what can you do to improve their results? Failure gets a lot of attention.

Some Calendar Statistics

We are currently in the process of redesigning and adding more functionality to the University wide calendar (more on this later).  We decided to run some statistics to see what people have used and to get a better understanding of our users.  Here are some interesting statistics we came across.

Statistics

136 separate calendars
30,804 total events among all sites
42,922 event occurrences (regular + recurring)
2,849 events emailed to friends

71% of events are recurring
41% of events are in the Student Center

14 events added per day on average (including weekends)

Most Used Calendars
6681 events by Student Center
823 events by Dean of Students Office
545 events by College of Fine, Performing and Communication Arts

First Event
CULMA’s  Open House and Annual Reception August 31, 2004

Most Recent Event
MAME Annual Conference
October 21, 2009

Most Recent Recurring Event
11th Annual Career Evening January 7, 2011

Date with the most events
94 Events – September 8, 2008

An insight into wayne.edu traffic

I have not talked a lot about the number of visitors our homepage (wayne.edu) gets on a monthly basis because it really didn’t cross my mind. I won’t go too in depth but I will give you insight into some general statistics and trends.

Stats from June 15 – July 15, 2009

Top Pages

top-pages

Of course the homepage is our most hit page but the second surprised me. I thought the future students would be #2 but probably because we are well into the semester current students are an overwhelming majority. Also these numbers were pulled from internal and external audiences.

Referring Sites

referrers

referrers-graph

A majority of our traffic comes direct, to me this is not that odd since this data set is from both internal and external audiences, most computers on campus have wayne.edu set to their homepage. I could be totally wrong about that but there is no way to test direct traffic. Search engines rank high over all, especially Google since we use that for internal site searches. Student voice is an interesting one, in the last few months the Dean of Students has been sending out surveys and they all forward back to wayne.edu, forcing a lot of traffic to our homepage.

Top Browsers

browsers

Internet Explorer is still a large majority of our traffic. Although the computers in the libraries all have Firefox, some students still click on that blue E out of habit (I have just sat in the library and watched behaviors before).

internet-explorer

Out of the Internet Explorer visits, here’s how it breaks down. These visit percentages are out of the 70.5% IE usage, not out of the total visits. Unfortunately IE6 still accounts for ~20% of our total traffic.

Operating Systems

operating-systems

Windows is still the clear winner in the OS category although Macintosh has been gaining. It is interesting to note that the iPhone and iPod account for almost 0.7% of our web traffic, more than double the Linux traffic. Adding up all the mobile devices it accounts for ~1% total traffic, it might not seem like a lot now but that number has been growing fast!

Overall nothing super surprising with the numbers, they are general stats so nothing to in depth with campaigns or targeted paths. The two large trends I wanted to point out where the decrease but still not present IE6 usage and the upward trend of mobile browsing.

"Schools & Colleges" vs "Academic Programs"

We don’t usually tell the community when we are doing A/B testing so the results don’t get skewed. One recent one has been running for the past few weeks and it looks like its not going to end up with a conclusive result so we are stopping the test right now.

“Schools & Colleges” vs “Academic Programs”

We are ever changing and refining the navigation for wayne.edu and over the past year of so many more universities have been changing their wording from “Schools & Colleges” to “Academic Programs”. That got us wondering, we thought academic programs had a more concise description of what is on the page behind the link. So we decided to try it out.

wayne.edu navigation transportation

Technical Results

May 21 – June 15, 2009 (3 weeks)
126,433 Page Visitors
20,644 Conversions
16.33% Conversion Rate

wayne.edu navigation results

Why was there no clear winner?

Two reason’s why I think no winner was found. The first is an issue on our end, we displayed the test for everyone, internal and external computers. Typically on campus staff/faculty have wayne.edu set as their homepage, they know the site and what they need to get to. Regardless what the link is called they will click it, this throws off the results since they are not our primary audience and they are not actually thinking about the click they are making.

The second is that someone looking for our list of programs has really only one route to get to it from wayne.edu and that is this link. It might take someone longer to figure out “Schools & Colleges” will give them a list of our programs but they still got there. They are adapting to our wording instead of ignoring our wording and using another method to find the information they need. I even wonder if we changed it to “What we offer” we would still get people clicking on it because they have a specific task in mind already.

We ended up just keeping Schools & Colleges for now

The thought is since there was no real winner although we love academic programs better we cannot justify the change. The university community has been voicestrous about any changes to wayne.edu and we want to make sure we have some real numbers to backup a navigation change.

Frankly the homepage navigation is not where we want it to be at all. It’s these types of tests that get us in the right direction on purpose instead of on a whim.

We are going to run another test on an unknown date for an unknown time with the lessons we learned from this test. Sorry we can’t tell you more but I am sure we will write about it afterward.

[Guest Post] Tracking Flash Interaction with Google Analytics

Over at .eduGuru they are searching for a new blogger. They took applications and had everyone write a guest post, they then let the community decide who the next blogger should be.

I was one of the applicants and wrote my post on Tracking Flash Interaction with Google Analytics. The goal was to not only give the readers an enjoyable article but also real tools that can be used on their site for free and back that up with an actual example.

So we only have one week to vote, below I have the link to all the guest posts and the link to vote. All the applicants did a great job! I encourage everyone to read all the articles and vote for who you think should be the next .eduGuru blogger.

Read All Guest Posts

Vote For Your Favorite

Voting closes Midnight Saturday November 29th.

Studying Students Social Network Usage

Previously had no concrete statistics about which social networks our current or prospective students use. Last week we ran some statistics for the visitors of the wayne.edu homepage to figure out which social networks they had visited.  We ran it on the weekend of May 31 to June 1st to get the largest percentage of external users as possible. We limited it to the homepage only, no other pages were tracking this information since current students often jump right to their email or Blackboard.

Our Method

We used a modified version of the SocialHistory JS created by Aza Raskin. The technique is pretty straight forward by creating an iframe and using the CSS a:visited to see if you have visited a site before or not.

Sites we looked for

Digg, Reddit, StumbleUpon, Yahoo Buzz, Facebook, Del.icio.us, MySpace, Technorati, Twitter, After5, MetroMode, Newsvine, Songza, Slashdot, Ma.gnolia, Blinklist, Furl, Mister Wong, Current, Menaeme, Oknotizie, Diigo, Funp, Blogmarks, Yahoo Bookmarks, Xanga, Blogger, LiveJournal, Last.fm, N4G, Faves”, Simpy, Yigg, Kirtsy, Fark, Mixx, Orkut, Upcoming.org, Google Bookmarks

Frankly some of those I had never even heard about but we tested for them any way.

Results

The first two graphs are percentages of all the users who visited the homepage. The rest of the graphs are only percentages of the users who visited one or more social networks before coming to wayne.edu.

In total the homepage was hit 25,927 times over the course of the weekend (due to an unexpected issue we did not track early Sunday morning from midnight to 5am), of those 15,600 had visited at least one social network before coming to wayne.edu

Internal v. External Users

Internal v. External Users

Our usual percentage is closer to 50/50 since all the computers in the libraries open to wayne.edu.

No Social Network Users v. Social Network Users

No Social Network Users v. Social Network Users

Number of Networks per User

Number of Networks per User

Popular Social Networks

Popular Social Networks

Top Network Combinations

Top Network Combinations

Ending Thoughts

Although these results are not 100% fool proof since the method we used is not a gaurentee usage of a site but just a visit to a site and it tracks mainly the sites homepage and login page. Some users may be using 3rd party applications primarily to interact with a site (Twitter for examle). We had a suspicion Facebook and MySpace would be most popular, we just didnt know which one would come out on top. It is pretty clear Facebook is the winner with MySpace coming in second, no other site stood up to those two. Having this concrete evidence was crucial for us to make future decisions based on what communities our prospective students are part of.