When I wrote about the Gartner report, readers commented on the fact that open source solutions were not part of the scope, even though their market share is increasing rapidly. The Forrester report has the same approach, except it includes vendors offering their products stand-alone as well as those with products integrated in portal/ECM solutions.

So why the exclusion of open source? Well, it appears difficult to decide on how to evaluate open source, especially when it comes to more advanced appliances.

Looking at the Forrester report, it includes some familiar conclusions but also a few new insights. Leslie Owen from Forrester concludes that “Google, Autonomy, and Microsoft are the most well-known names; they own a large portion of the existing market”. Hence, these vendors are still standing strong, even though they are challenged in various areas.

More surprisingly, some niche players get higher scores than the giants in core areas such as “Indexing and connectivity”, “Interface flexibility” and “Social and collaborative features”.

Vivisimo is seen as somewhat of a leader (with a slightly lower score on Mobile support and Semantics/text analysis). In the Gartner report, Vivisimo was excluded from the information access evaluation due to the fact that they were ”focusing on specialized application categories, such as customer service”.

An interesting reflection from Forrester is that “in the next few years, we expect prices to rise as specialized vendors wax poetic on the transformative power of search in order to distinguish their products from Google and Microsoft FAST Search for SharePoint”. On the Nordic market, we have not seen a shift to such a strategy, but rather the opposite, since open source (with zero license fees) is becoming accepted in an Enterprise environment to a larger extent.

The vendors that provide integrated solutions (to CMS/WCM etc) still remains strong, whereas the stand-alone solutions becomes exposed to completion in new ways. It will be interesting to follow the US and Nordic market to see how this evolves within the next year. It might be that the market differs when it comes to open source adaption.

If you wish to read the full report it can be downloaded from Vivisimo through a simple registration.
To get a complete overview of vendors, I recommend reading both the Gartner and Forrester report.

First, I was intrigued by the session on evaluation for IR and especially the topic of Croudsourcing. In my opition, the paper A Methodology for Evaluating Aggregated Search Results, which also got the prize for best student paper, was among the most pedagogically presented ones. It deals with the task of incorporating search results from a number of different sources, called verticals, into Web search results. By using a small number of human judgements for a given query the authors present the way to evaluate any possible permutation of verticals in the result presentation. I think that this methodology should be adopted in the world of Enterprise search, since it is exactly there where we crawl, index and present information from a number of different sources – Web, databases, fileshares, etc. The prerequisites are really minimal and low cost but the return value, the user experience, seems quite high.

Amazon Mechanical Turk, or the Artificial Artificial Intelligence, which is the marketplace for Croudsourcing, provides a way for a ridiculously small sum of money to perform evaluation, relevance assessment or any task for which you would need humans to give you some judgements. Leaving aside ethical issues, two papers in the conference presented ways of how you can utilize this service for some IR tasks.

Evgeniy Gabrilovich from Yahoo! Research, who won the Karen Sparck Jones award for 2010, gave a very interesting keynote talk on Computational Advertising. Up to now, it has never struck me how hard advertising in Information Retrieval systems is actually. I liked one of his points on the future of Ads – by using product feeds, one can automatically create product description via Text Summarization and Natural Language Generation and index this, thus avoiding bid words.

Another interesting and very pedagogically presented paper was about the gensim package by Radim Řehůřek. I definitely think we can use it in some of our projects. In general, text categorization and IR for social network were the dominant tracks. In one of the social networks tracks, Oscar Täckström presented a neat way of discovering fine-grained sentiment where some coarse-grained supervision is available. It really hooked me on trying it for any of our customers where sentiment analysis is required.

Thorsten Joachims, the last of the keynote speakers, gave a very inspiring talk on The Value of User Feedback. He put forward the idea of designing retrieval systems for feedback. In stead of just looking at the clicklogs post factum one can think of a system which uses the clicks feedback to learn, thus creating a better ranker for a given query and a given user need. In a single session, we can use click feedback to disambiguate the query and deliver results on the run which are of immediate benefit to the users.

Unfortunately, I guess I could have missed other interesting presentations but with two parallel sessions and several workshops there was a limit to what I could devour. What surprised me though, was that there were very few papers by the industry. We do try to solve exactly the same problems and tackle the same issues as academia. We, at Findwise, have constantly flagged the huge benefit of good, relevant Metadata for the task of achieving better search performace, which was also touched upon in the paper “Topic Classification in Social Media using Metadata from Hyperlinked Objects”.

It was really great to visit Dublin and attent ECIR 2011. It was an inspiring conference and I do believe that at next ECIR we, from Findwise, can be on the podium, sharing our knowledge and hands-on experience on Enterprise search and IR.

Sláinte!

• Investigate how location-based information (as opposed to fixed locations) could be connected to search results
• Improve quality of location-based information by considering the course and velocity of the user

To start with, I created an iPhone application with a location-based reminder system. The reminders described location constraints and users could create reminders with single locations (at home) or groups of locations (at any pharmacy). To find these groups of locations, the system searched for locations with associated information (like nearby pharmacies) and delivered this information without users having to click Search repeatedly.

This is an unusual approach to search as the user is passive, instead the system is performing searches for the user. However, to make search results relevant one has to add contextual constraints to describe when, where and to whom a piece of information is relevant. When all constraints are met, information should be relevant. If not, the system lacks some crucial contextual constraints.

When search is automated, the importance of relevant search results increases and the more you know of the users world, the better you can adjust the results. However, traditional search can also benefit from contextual information. It can be used as a filter where search results that are irrelevant in the current context are removed. Alternatively it could be a part of the relevance model, improving search results by reordering them according to context. Hence, whereas automatic information delivery is probably undesirable for many types of information – contextual constraints can still be of good use!

The people who tested my application created 25% of their reminders as groups of locations and found it useful as it helped them find places they weren’t aware of, facilitating opportunistic behavior. The course and velocity information reduced the number of false-positive information deliveries. Overall, the system worked well as a niche product.

At the end of last year Gartner decided to retire their old “Information Access Quadrant” and introduce “Enterprise Search MarketScope” due to a more mature market. A number of vendors (such as Vivisimo and Recommind) were removed, in order to exclude those whose businesses were not entirely search driven.

The evaluation criteria’s for MarketScope cover: offering (product) strategy, Innovation, Overall viability (business unit, financial, strategy, and organization), Customer experience, Market understanding and business model.
To summarize: the criteria’s are to a large extent the same, but the two areas “overall viability” and “customer experience” are weighted higher than the rest. This is most likely a result of the last years discussion around user friendly interfaces, easier administration and the fact that some customers have suffered quite bad when vendors do not survive (one example in Northen Europe is the Danish vendor that went bankrupted for some time)

The yearly fight between the three leaders; Microsoft, Endeca and Autonomy has been somewhat disrupted and Microsoft, Endeca and Google are now seen as the leaders.
Microsoft has got a very broad product line, which stretches from low-price and less functionality to Enterprise Search built on the former FAST technology. Endeca follow the same trend, as Gartner puts it their “products (are) intended to serve organizations seeking to develop general search installations..(..) broadly applicable for a variety of different search challenges”.
In the old quadrant, Google remained a “challenger” for quite some time – but never made it to the “leaders” corner. Ease of administration and “user friendly” are two words that keeps being repeated. That, in combination with a profit of $ 7290000000 during the last quarter of 2010 makes Google a player that easily can continue to develop their Enterprise business.

Gartner's MarketScope for Enterprise Search

Autonomy should still not be disregarded, the main reason for it falling a bit behind the three others seem to be conquerable problems with support and pricing transparency. It will be interesting to see how Autonomy chooses to handle these issues during 2011.

To put it short: the new MarketScope is good reading with quite few surprises. If you wish to get a better understanding of the development going on at the different vendors, start with Gartner and continue to search among our blog posts.

We use a work-task based research approach where we study information practices – that is, the normalized ways we use to recognize information needs, look for information, and how it is valued and used. By studying such practices in real-life work tasks, we can outline the role that a search engine plays in relation to other work tasks as well as to other ways of finding information. In short, being engaged in a creativity-oriented work task initiates different types of information practices compared to the practices we use in everyday, routine-based work tasks …

The creativity-oriented work tasks involve a dimension of innovation, and concepts such as learning and development are often used to describe these activities. Uncertainty is something that is associated with curiosity and may be seen as a driving force behind information seeking. Information that is rich in nuances and that offers different, even contradictory explanations or descriptions is usually appreciated, and the task outcome is only vaguely discerned at first. Routine-oriented tasks, on the other hand, are focused on increasing effectiveness and reducing uncertainty as quickly as possible in the task outcome, which itself may be sketched out relatively clearly from the beginning. Information seeking is often directed to readily available facts. All this means that a search engine must support a variety of information practices at any given workplace!

The “we” in this project is myself together with a Findwise colleague Henrik Strindberg. The project is financially supported by the Swedish Foundation for Strategic Research, and while I am not working with the present project I am employed by the University of Borås.

Just now I am finalizing a presentation of the project for the ICKM conference in Pittsburgh, PA, USA, next week. The presentation is entitled “Interrelated use and value of information sources”, and will be available through the conference proceedings in due time.

Very exciting … and while there I will also attend the board meetings of the ASIS&T’s Board of Directors as a newly appointed Director-at-Large. Very exciting, too!

The 73rd Annual Meeting of ASIS&T focuses on “Navigation Streams in an Information Ecosystem”.

This years keynote speaker was Dan Russell, a researcher from Google. He talked about Search Quality and user happiness; Why search is easy and hard. The point I found most interesting in his presentation was how improvement is not only needed when it comes to tools and data but also improving the users’ search skills. My own experience from various search projects is similar; users are not good at searching. Even though they are looking for a specific version of a technical documentation for a specific product they might just enter the name of the product, or even the product family. (It’s a bit like searching for ‘camera’ when you expect to find support documentation on your Dioptric lens for you Canon EOS 60D.) So I agree that users need better search skills. In his presentation Russell also presented some ideas on how a search application can help users improve their search skills.

Search is both easy and hard. Perhaps this is one of the reasons for the introduction of the HCIR Challenge as a new part of the workshop . From the HCIR website:

The aims of the challenge are to encourage researchers and practitioners to build and demonstrate information access systems satisfying at least one of the following:

• Not only deliver relevant documents, but provide facilities for making meaning with those documents.
• Increase user responsibility as well as control; that is, the systems require and reward human effort.
• Offer the flexibility to adapt to user knowledge / sophistication / information need.
• Are engaging and fun to use.

The winner of the challenge was a team of researchers from Yahoo Labs who presented Searching Through Time in the New York Times. The Time Explorer features a results page with an interactive time line that illustrates how the volume of articles (results) have changed over time. I recommend that you read the article in tech review to learn more about the project, or try out the Time explorer demo yourself. You can also learn more about the challenge in this blog post by Gene Golovchinsky.

All the papers and posters from the workshop can be found on the new website.

A lot of sites are improving their search experience through the implementation of faceted search. However, very few successfully integrate faceted search and browsing on their site. Searching and browsing are treated as two separate flows of interaction instead of trying to combine them which would provide the users with a much better experience.

That is why I was glad to learn about an idea from Greg Nudelman which he presented in his session at the IASummit which I attended last week. In his session Greg introduced his idea about Integrated Faceted Breadcrumb. According to him breadcrumbs are intuitive, flexible and resourceful and they are design elements that don’t cause problems but simply work. To test his idea he conducted usability tests on a prototype using the Integrated Faceted Breadcrumb. According to his evaluation the integrated faceted breadcrumb has a lot of advantages over other faceted solutions:

1. Combine hierarchical Location & Attribute breadcrumbs
2. Use Change instead of Set-Remove-Set
3. Automatically retain relevant query information
4. Label breadcrumb aspects
5. Make it clear how to start a new search
6. Allow direct keyword manipulation.

I find this idea interesting and I am currently thinking about whether it could be applied into one of my own projects. (According to Greg it has not been implemented anywhere yet even though the findings from the usability testing were positive.) However I wonder if this is a concept that works well only for sites with relatively homogeneous content or if it would also work on larger collections of sites such as intranets? Can it be used in an intuitive way with a large number of facets and can it cope with the use of more complex filtering functionalities? For some sites it might not be the best idea to keep the search settings when the user changes search terms. These are some things I would like to find out. What do you think about this? Could you apply it to your site(s)? I recommend that you have a look at Greg Nudelman’s presentation on slideshare and find out for yourself. You can also find an article about the Integrated Faceted Breadcrumb on Boxes and Arrows. I look forward to a discussion about whether this is any good so write me a comment here at the findability blog or find me on twitter.

Such functionality includes faceted search which serves as filters to narrow a search. These facets might need to be different based on what kind of role, department or background users have. Developers can create a set of facets and let the users choose the ones that satisfy their needs. Search driven portals is a great tool to make sure that sites don’t get flooded with information as new functionalities are developed. If a new need evolves, or if the provider comes with new ideas, the functionality is put into new searchlets which are deployed into the searchlet library. The administrator can broadcast new functionality to users by putting new searchlets on the master page, which affects every user’s own site. However, the users can still adjust new changes by removing the new functionality provided.

Search driven portals opens new ways of working, both in developer and usage perspective. It is one step away from the one size fits all concept, which many sites is supposed to fulfill. Providers such as Findwise can build a large component library which can be customized into packages for different customers. With help of the searchlet library, web administrators can set up designs for different groups, project managers can set up a project adjusted layout and employees can adjust their site after their own requirements. With search-driven portals, a wider range of users needs can more easily be covered.

According to Shah and Marchionini users often search for the same things. They reuse their previous queries e.g. search for the same things multiple times. Users use their previous searches to refind information and also to expand or further filter their previous searches by adding one or more keywords. There is also a significant overlap between what different users search for suggesting that users have a tendency to express their information needs in similar ways. These results support the idea that query suggestions can be used to help users formulate their query.  Yahoo and YouTube  are two of the systems that uses this technique, where users get suggestions of queries and how they can add more words to their query based on what other users have searched for.

Diane Kelly concludes that users use query suggestion both by typing in the same thing as shown in the suggestion and by clicking on it. Users also tend to use more query suggestions when searching for difficult topics. Query suggestions help users get “unstuck” when they are searching for information.  It is however hard to know whether query suggestions actually return better results. The users expectation and preferences do have an effect on user satisfaction as well. User generated query suggestions are also found to be better than query suggestions generated by the search system. So the mere expectation that the query suggestions will help a user could have an positive effect on his or hers experience…

Query suggestions are meant to help the users formulate a good query that will provide them with relevant results. Query suggestions can also work as with yahoo search where query suggestions both suggest more words to add to the query but also provides the users with suggestions for other related concepts to search for. So searching for Britney Spears will for example suggest the related search for Kevin Federline (even though they are now divorced) and searching for enterprise search will suggest concepts such as relevance, information management and off course the names of the different search vendors.

If you apply this to the enterprise search setting the query suggestion could provide the user with several different kinds of help. Combining the user’s previous searches with things other users searched for but also providing suggestions for recommended queries or concepts. The concepts will be high quality information and suggestions controlled by the team managing the search application. It is a way of combining quick links or best bets with query suggestions and a way to hopefully improve the experienced value of the query suggestions. The next step then is to work with these common queries that users search for and make sure that they return relevant results, but that is an entirely different topic…

The threat to the industry from low cost countries, which no longer only compete just on low cost, but also with high quality and competence, forces Western organizations to develop new strategies to sustain their growth and competitive advantage. Cutting margins in order to compete with low cost countries is a downward spiral. Instead changing the model, to be able to provide knowledge and holistic understanding of customers needs and the ability to rapidly deliver a complete solution is now becoming the key competitive advantage. This however requires investment in IT and knowledge exchange tools. By moving away from selling physical products and components to solutions higher margins are possible because more business value is exchanged in the transactions.

The organizations adapting to this change, are identifiable by the fact that they consider knowledge and information as corporate assets – treated and cared for as any other asset. One example is the Swedish company SKF Group whose new vision is the “Knowledge Engineering Company” where the company going through a change from component supplier to a holistic supplier of both products and services.

A key success factor in this transformation from products to solutions is that the supply of knowledge and information to the employees is effective, easy to use and complete. The organization succeeds in providing that extra value, thereby allowing higher margins. Historical key performing indicators (KPIs) such as factory output, reduction of defects and increasing of quality, are dealing with physical production efficiency to ensure as little cost per unit manufactured and as high quality as possible. Individuals are used to measuring these KPIs and provide a way to manage the operational production processes. The turnover and efficiency of information and knowledge exchange lacks these models and measurement tools, thereby not allowing them to be managed. What you can’t measure, you can’t manage, or improve..

One technical solution which has the capabilities to enable complete, rapid and reduced turnover time for knowledge and information exchange, is Enterprise Search. This has been recognized by the Swedish Foundation for Strategic Research, which granted Findwise AB research funds to tackle this problem in cooperation with the College University of Borås in the Strategic Mobility program. The funded project will study the usefulness and value of a well functioning search engine for work-related information use. It will also identify performing indicators for information and knowledge exchange through search and to achieve results that systematically will illustrate the quantitative direct effects together with soft indirect effects.

The project will start early 2010 and run through the entire year. As part of the project, Dr. Katriina Byström will join Findwise and work together with Findwise employees in this joint research project. Findwise customers are invited to participate in the project and will have the availability to influence its direction. For more information contact Henrik Strindberg.

About Dr. Katriina Byström
Dr. Katriina Byström is an associate professor in the Swedish School of Library and Information Science at the University College of Borås & Goteborg University, Sweden. She is one of initiators and director of the IA bachelor’s programme at Swedish School of Library and Information Science, and she is a chair for the programme with teaching involvement broadly across the curricula. Furthermore , Katriina is associate editor and co-founder at the Journal of Information Architecture. Katriina’s degree is in information studies, and her research focus on task-based information seeking, information retrieval and information architecture.