A search box; a search button; and ten blue links – what could possibly be missing from this equation? With the volume of information within enterprises growing exponentially, it is also becoming harder to sift through the noise to find needed information. Enterprise search is a critical tool for retrieving and uncovering relevant content. But when search results don’t return the desired content, frustrated users are driven to try out yet another search engine; continuously repeating the same cycle, also known as The Enterprise Search Cycle.

BA Insight’s Principal Search Architect, Mike Himelstein, recently explored the enterprise search cycle and the drivers behind successful search. Why is search broken? What are the steps organizations can take to eliminate the cycle of broken search? Before embarking on the next search journey, companies must answer a set of questions to understand how to make search work in their organization.

Download the presentation, and read more about taking the path to ultimately transforming the search experience at your organization.

Jeff Fried, BA Insight’s CTO is a worldwide speaker and writer in the industry. He holds 15 patents and has written over 50 technical papers during his career- including our ever-popular Essential Guide to Enterprise Search in SharePoint 2013. During ESS, he took a deep dive into some of the most practical knowledge that search consultants and users need in order to optimize the use of the software. You will find a link to all the presentations from the Enterprise Search Summit Workshop: Architecting Optimal Search Strategies With SharePoint 2013. These invaluable guides provide in-depth practical skills displayed in a visual, and comprehendible manner.

A few questions you can expect to be answered:

• What is SharePoint 2013 search?
• What is the business value of search?
• How do I make search work?
• What are some key concepts of standalone vs. integrated search?
• How can I surface information from other enterprise systems using search?
• What are the components of an enterprise search strategy?

And much more…
Download here

Click here to learn more about expertise finding and the techniques to empower your people through social search in SharePoint 2013.

Moving to Office 365 has many benefits, including low entry costs, a financial shift from CAPEX to OPEX and improved business agility. It is truly an exceptional tool that can help your business reach new heights.  That said, I have one warning for those of you interested in moving to 365: be careful to avoid what I call the “Beatlemania” effect.  

It is very easy with any new cloud technology to get swept up in the frenzy and swoon over the agility and scalability that the cloud can provide. While moving to the cloud may want to make you twist and shout, it is good to be aware of the potential limitations of the new solution and what you can do to address them.

Some initial challenges associated with Office 365 are related to the sometimes complex nature of integrating cloud-based and on premise systems.  Limitations include:

1. Query  suggestions and related queries
2. FAST-specific features even though FAST isn’t available on Office 365
3. The abilities to do advanced content processing and crawl external content

Cloud/on premise integration challenges are not unique to Office 365. In fact, there are similar limitations with other applications throughout the cloud. A recent survey (see the chart below) of companies moving parts of their business into SaaS platforms, such as Salesforce.com, showed that the ability to customize and build on platforms and the ease of integration with on-premise systems are the weakest points for SaaS overall.  Not quite the sunshine daydream you imagined.

The first two sets of limitations – those related to queries and those related to FAST-specific features – will fade away with future releases.  The third set of limitations, however, will not disappear any time soon – and presents the following set of challenges for enterprises:

• Inability to manage and navigate their organization’s information
• Navigating data that is trapped in silos
• Satisfying a wide range of user requirements across diverse systems and diverse roles

Part of the challenge is that the volume of data in the enterprise is growing at an explosive rate.  According to estimates, the volume of business data worldwide, across all companies, doubles every 1.2 years. (Source)  While data is growing in volume, enterprise data is increasingly growing in unstructured content and number of silos.   The average enterprise of 1,000 – 10,000 employees has more than 150 different systems practically ensuring that data will remain siloed by application, line of business, or even business user.

The inability to bridge these silos can lead to poor or incomplete data which can cost businesses 20%–35% of their operating revenue. (Source)  Bad data or poor data quality costs U.S. businesses $600 billion annually. (Source)  Office 365 is a great example of your content can get trapped – you can get content inside of Office 365 but not outside of it.

Enterprise Search is your ticket to ride.  Search can help with these challenges by providing a unified view of your information across silos.  After working for more than eight years to help organizations bridge their data silos, lately we’ve seen additional complications related to hybrid solutions and the need to integrate online information services with on premise services.  Hybrid models often occur because building to the Cloud is rarely a matter of flipping a switch, especially for larger enterprises. It is a migration process, this holds true for Office 365 and a number of other business critical applications.

While Microsoft gives companies the flexibility by supporting an on-premise and Cloud hybrid,  the ability to search across both instances and present the information in an actionable format is hindered because it is not fully integrated.  Listen to our latest webinar, Integrating Office 365 with the Rest of Your Enterprise: Best Practices and Pitfalls, for strategies and tools to help you achieve a unified view with search for Office 365.

Read previous posts:

Office 365 Blog Series: Concepts and Techniques for Integrating Office 365 with the Rest of Your Enterprise

Today, I’ll give a quick refresher course, with help from some information published by Richard Harbridge, and will go over the benefits and tradeoffs of the Cloud and Office 365 and how you can use these solutions effectively. To kickoff this series, I’ll also discuss the most common pitfall when using Office 365: the fact that it creates information silos within your organization. When only a portion of an enterprise is running on Office 365 or only a portion of your content is in the Cloud, the need arises to bring those separate silos together. Search technology in general, and the Search capacities built into SharePoint in particular, can be used to bridge those silos and provide your users with a unified view of information, regardless of where it resides.

At the end of this Office 365 Blog Series, I will give you a set of techniques for providing a unified view of information for your Business Critical SharePoint, highlight some of the capabilities BA Insight brings in this regard and talk about best practices to help you go from crawling to flying, one step at a time.

The Cloud 101

Allow me to start with a reminder of basic concepts about the Cloud, simply because the Cloud is a very hyped and often misunderstood subject. As you see in the image below, there are three main kinds of Cloud services available today:

• Software as a Service (SaaS)
• Platform as a Service (PaaS)
• Infrastructure as a Service (PaaS)

The further down we move on this chart towards infrastructure (which is essentially running VM’s on a data center in the Cloud), the more you are able to customize and the more you have control. At the same time, the further up we move on this chart, the less complexity you will have in the product. With control and customization comes complexity. By this rule, Software as a Service, at the top of the chart, provides less complexity for its users but is also the least adaptable of the Cloud services, whereas Infrastructure as a Service, at the bottom of the chart, is the least simple but the most adaptable Cloud service.

Microsoft has a wonderful set of offerings that truly cover the whole spectrum. Windows Azure, which began as Platform as a Service, has now added capabilities for Infrastructure as a Service, which is very exciting.

Office 365, the focus of this blog series, is Software as a Service, and we’ll get into that.

Customization Versus Simplicity

As I mentioned above, on SaaS, you have the least customization capabilities of the three Cloud options but you also have the most simplicity. There are a couple of things to point out in terms of this tradeoff between customizability and simplicity. The first is that in SharePoint Online or Office 365, there isn’t an ability to administer the farm. You, as a tenant (it’s called a multi-tenant service), have the ability to administer site collections independently, and I’ll explain later what that means from a control perspective.

From a developer perspective, in order to be sure that Office 365 runs in multi-tenant mode without the tenants impacting each other, add-ons to SharePoint are either client-side add-ons or things that run in what’s called the Sandbox. The Sandbox is a great capability of SharePoint 2010 and allows you to do many things. However, it also has limitations. The primary limitation is the ability to access resources that are on the local server. In particular, for Search, there is no way to index into Office 365 content sources that are outside of it.

Moving to Office 365: Key Benefits

The first benefit of Office 365 is that it is low cost and fairly easy to get going. Our company, BA Insight, uses Office 365 and it has been a very cost-effective solution for us. The second benefit of Office 365 is the financial shift from upfront capital expenditure to something that is a predictable, regular and ongoing operating expenditure (CAPEX to OPEX). The third benefit, and in my opinion the best, is improved business agility. With Office 365, you don’t have to stand up your own servers, install your own software or run your own operations. You can focus on core competencies and benefit from other companies providing a very scalable, flexible platform. Offered enterprise platform features include:

• Premium anti-spam and antivirus protection provided by multiple virus scanning engines
• Data replication in geo-redundant datacenters to protect against datacenter-wide failures
• Risk mitigation multi-dimensional approach to help safeguard services and privacy of data
• Compliance with ISO 27001, SAS 70 Type I, FERPA, HIPAA, FISMA, EU Safe Harbor Seal
• Backing by a 99.9% financially backed Service Level Agreement

Office 365 is a Microsoft platform that is being continually updated. In later posts, I’ll touch on the upcoming SharePoint 2013 and Office 2013 abilities that were recently announced. For this blog series, I will focus on the current edition of Office 365 – specifically the server side of it – Exchange, SharePoint and Lync. The basic idea is that these are the same applications you are used to that are now software you can use in the Cloud. They are running on a reliable, redundant, geo-distributed and certified platform. This ensures that the security and reliability of all of the Cloud services, and Office 365 in particular, are very strong.

To get ahead on the topics that I will be discussing in the future posts, and for more information on how to transition to the Cloud and prepare your SharePoint for the move, watch my webinar, now available on demand: Integrating Office 365 With the Rest of Your Enterprise – Best Practices & Common Pitfalls.
 

The Problem of Siloed Data

Many organizations utilize a variety of technology platforms and applications, often built for specific vertical business functions and business roles.  (Some common examples include  Product Lifecycle Management [PLM] for design and engineering, Customer Relationship Management [CRM] for sales and service, and Enterprise Resource Planning [ERP] for finance and human resources.) 

Applications and platforms like these can benefit enterprises by supporting structured decision making.   However, due to their various security architectures and UIs (and overall lack of interoperability), this diversity of platforms often creates ‘information silos’.  These silos make it difficult (and sometimes impossible) for users to easily access the information they need to work efficiently and drive collaborative decision-making across business units.  The solution to this problem is to surface this enterprise information through a single presentation and collaboration layer.  (And it just so happens that the most feature rich, scalable, and cost-effective platform out there is SharePoint!) 

Achieving Information Integration with Search

One of SharePoint’s core capabilities – Search – is a powerful tool for unifying information access across the enterprise.  The best-in-class search engines provided by FAST Search for SharePoint (FS4SP) and SharePoint 2010, as well as SharePoint 2013 – when coupled with BA Insight’s SharePoint Connectors – enable the secure search across all line of business systems.   SharePoint’s search index then serves as an agile ‘enriched data warehouse’ for the enterprise…. the single source for finding business-critical information.

With this Search-based information architecture in place, organizations can then quickly deploy Search Applications atop SharePoint.  These applications leverage SharePoint’s enriched search index (which, with the help of BA Insight can include, interrelate, and normalize the content & metadata from across all enterprise systems), to empower users to not only find the information they need, but also to engage this content with SharePoint’s expansive collaboration toolset, including workflow, records management, and enterprise content management.   

Become Your Company’s Information Champion!   Take Advantage of a Free SearchPoint Strategy Session

BA Insight’s participation in Microsoft’s Business-Critical SharePoint Program is further testament to our commitment to helping organizations address their data access challenges and take full advantage of their SharePoint investments.  In support of the program, BA Insight is now offering organizations complimentary SearchPoint Strategy Sessions, half-day consulting engagements designed to help both IT and business stakeholders:

• Better understand all that is made possible by using SharePoint Search across the enterprise.
• Identify where and how Search-based applications can be deployed to accelerate productivity.
• Learn how to take full advantage of existing investments in Microsoft technologies.

BA Insight’s Search experts, including Jeff Fried, Agnes Molnar (SharePoint MVP), Mike Himelstein, Rem Purushothaman and Anthony Butcher, are recognized industry experts and thought leaders with years of experience and hundreds of successful Search implementations.   If you would like to meet with one of our search experts, to help develop your information access strategy and explore how to take better advantage of SharePoint’s capabilities, let us know.

Click here for more information about our SearchPoint Strategy Session.

It works as advertised, but seriously messes up results ranking. For example, if you search for Anjaya: people with names like Sanjaya, Ranjana and all other variations show up on the first page of results while the persons with name Anjaya are buried in 2^nd, 3^rd, or 4^th pages of results.

This has been a serious issue for many customers I have worked with and they have always asked me to disable phonetic search.

The SharePoint 2010 People Search webpart is sealed, so there isn’t an easy way of disabling it. The approach I take is to create a custom relevancy ranking model for People Search. In the ranking model, remove the property “pronunciations” so that phonetic matches are ignored. Finally use the custom ranking model. Here are the detailed steps:

• Take the default ranking model XML and remove “Pronunciations” Query dependency feature. Modified XML looks like this:

<?xml version=”1.0″ encoding=”utf-8″?><rankingModel name=”CustomPeopleRanking” id=”70A04609-5BA2-4B9B-9E81-EE6CC4BD6519″ description=”Custom People Ranking” xmlns=”http://schemas.microsoft.com/office/2009/rankingModel”>
<queryDependentFeatures><queryDependentFeature pid=”177″ name=”RankingWeightName” weight=”0.5″ lengthNormalization=”0″ /><queryDependentFeature pid=”19″ name=”PreferredName” weight=”1.0″ lengthNormalization=”0″ /><queryDependentFeature pid=”24″ name=”JobTitle” weight=”2.0″ lengthNormalization=”0″ /><queryDependentFeature pid=”39″ name=”Responsibilities” weight=”1.0″ lengthNormalization=”5″ /><queryDependentFeature pid=”179″ name=”RankingWeightLow” weight=”0.2″ lengthNormalization=”5″ /><queryDependentFeature pid=”175″ name=”ContentsHidden” weight=”0.1″ lengthNormalization=”5″ /><queryDependentFeature pid=”35″ name=”Memberships” weight=”0.25″ lengthNormalization=”5″ /><queryDependentFeature pid=”178″ name=”RankingWeightHigh” weight=”2.0″ lengthNormalization=”0″ /></queryDependentFeatures></rankingModel>

• Copy the ranking model XML to a notepad, enable “word wrap” and copy the xml.
• Run this powershell command to create a custom ranking model

            Get-SPEnterpriseSearchServiceApplication | New-SPEnterpriseSearchRankingModel

            Paste the XML when prompted.

• Create a custom webpart that will force all search locations on a page to use a custom ranking model. This is also described in this blog post: http://blog.petergerritsen.nl/2010/10/11/let-the-sharepoint-search-web-parts-use-an-other-ranking-model/

public class VisualWebPart1 : WebPart

    {

        // Visual Studio might automatically update this path when you change the Visual Web Part project item.

        [Personalizable(PersonalizationScope.Shared)]

        [WebBrowsable(true)]

        [WebDescription("RankingModelId")]

        [WebDisplayName("RankingModelId")]

        [Category("Custom Settings")]

        public string RankingModelId { get; set; }

         protected override void OnInit(EventArgs e)

        {

            try

            {

                Microsoft.Office.Server.Search.Query.QueryManager qm = Microsoft.Office.Server.Search.WebControls.SharedQueryManager.GetInstance(this.Page).QueryManager;

                foreach (LocationList ll in qm)

                {

                    foreach (Location l in ll)

                    {

                        try

                        {

                            if (!string.IsNullOrEmpty(RankingModelId))

                            {

                                l.RankingModelID = RankingModelId;

                            }       

                        }

                        catch { }

                    }

                }

                base.OnInit(e);

            }

            catch { }

        }

    }

• Drop your custom webpart to a people search page. Edit the webpart and specify the custom ranking model GUID under Custom settings>RankingModelId. In this case the id is “70A04609-5BA2-4B9B-9E81-EE6CC4BD6519”.

That’s it. People Search core results webpart will use the custom ranking model you just deployed and results will not be based on phonetic matches anymore.

Would love to hear if you think you’d choose to disable Phonetic Search in SharePoint 2010 or take advantage of it. What are the Pro’s and Con’s in your case?

Enterprise search technology is incredibly useful and powerful, but very few people understand how to apply it well. Perhaps because search looks so simple on the outside, or perhaps because of the universal familiarity with web search, enterprise search is generally poorly understood. There are many misconceptions and misunderstandings about enterprise search, and in many organizations the first comment you hear about an intranet is, “the search is useless.”

We can do search better. There is no silver bullet or secret sauce, but if you follow some basic ground rules and utilize some proven, practical techniques, you can make search something that people not only use but love.

“With great enterprise search, organizations can quickly respond to market changes, innovate and accelerate their time to market. Without good search, people drown in their own information,” writes Jeff Fried, CTO of BA Insight in his new article “The Search Journey”, “When search works well, people use it. Improving search quality will increase its utilization and make people more effective at their jobs.” In the first installment of his regular TechNet column, A View from the Crawl Space, Fried gives practical advice for how to make search the useful, powerful tool it can be. Fried explains that search is not a one-size-fits-all endeavor, and while this means that enterprises are able to create a solution perfectly tailored to their unique needs, it also means that there is no secret sauce or silver bullet strategy for search adoption. There are, however, certain key common denominators.

Fried outlines three ground rules that apply in all cases and should be heeded by any organization, regardless of where they are in the “search journey”. The three rules are simple:

• Start projects with envisioning and out-of-the-box search
• Make search administration someone’s job
• Enlist the users.

Visit TechNet’s Enterprise Search Research Center or read Fried’s full article “The Search Journey” (PDF) to learn more about the three rules and other practical advice for building great search.

By Stephen E. Arnold,

Seven years ago, a child was swept from her home in the 2004 Indian Ocean tsunami. In December 2011, she was reunited with her parents. Serendipity, not a search technology, delivered this result. Unable to remember details of her past, the teenager was identified by birthmarks and a mole.

In most organizations, information tsunamis rage each day and locating information often bears striking similarities to the story of Meri Yulanda in Indonesia. Chance and elbow grease often deliver the needed information. Technology often gets in the way of making connections, of solving tough, real-life findability problems.

Countless organizations rely on Microsoft SharePoint. There are millions of users and almost as many blog posts, technical articles and consultants to solve tsunami-scale information problems. SharePoint began life as a content management system. In 2001, fresh from repositioning its content management solution and its acquisition of NCompass Labs, SharePoint Portal Server became available. The foundation of today’s SharePoint solution was in place. SharePoint 2010 offers an incredible range of features and functions. Those include support for public or private websites, community and collaboration services, content management, search and retrieval, business intelligence and “composites,” which I admit I do not fully understand.

What makes SharePoint such a popular solution? Why does the system spawn so many third-party fixes, enhancements and add-ins? How expensive is the system when fully deployed? I explored those and other questions with the founder of BA Insight, a successful Microsoft certified vendor squarely focused on the exploding SharePoint market. BA Insight is in the knowledge management business, and its core product is Longitude Search. A licensee struggling to deliver information access to SharePoint users installs Longitude and uses that system to add functionality to SharePoint and other enterprise systems. […]

Pinpointing information faster

BA Insight and its Longitude service grew out of Mounier’s firsthand experience with enterprise systems and Microsoft SharePoint. Longitude Search evolved from a large search project Mounier completed for a Fortune 100 energy company. He explained, “The data integration challenges we experienced were not unique to me and my team. It was evident to us seven years ago and even more so today, that enterprise search technology would be the technology foundation for more cost-effective, unified information access platforms. This is the situation today and is due to the fact that search platforms have significantly matured and can be connected to disparate data sources ranging from entirely unstructured content to highly structured data.”

Longitude adds some useful features to the SharePoint system. When I get a results set from BA Insight’s system, I can preview a document without opening another application such as Adobe Acrobat, which can be sluggish. The goal of BA Insight’s approach is to reduce the time required for a SharePoint user to pinpoint the information needed to complete a work task.

Like a number of other companies, BA Insight uses the phrase “search-based applications” (SBAs) to describe the approach the company takes. The idea is that employees have to find information in order to do work in a modern organization. Search, therefore, is not a sideline activity. Search sits at the center of an organization’s enterprise systems. The marketers have seized upon two buzzwords to describe the value of the search-based application approach: agility and adaptability.

Mounier explained, “Agility is the rapid deployment and immediate value one gets out of the box. Agility turns out to be the key differentiator with other types of information access applications, such as business intelligence (BI), which traditionally require a larger upfront investment and extensive user training to realize measurable return on investment. Adaptability in the context of SBAs characterizes the technology’s ability to automatically tailor to the industry, the department or the function a user is part of, without having, for instance, to predefine an industry or department specific taxonomy.  As a contrast to SBAs, a BI cube or BI report is essentially set in stone, expensive to change and maintain. Similarly, a conventional enterprise search tool does not learn from users’ interactions, but often relies entirely on a predefined taxonomy that is even more expensive to change and maintain.”

The explanation evokes a number of other buzzwords, and I am not sure if procurement teams know what a “taxonomy” and a “BI cube” are. When I expressed skepticism at those definitions, Mounier replied, “Our technology and our SBAs learn and improve based upon every user interaction and dynamically present the most relevant information context, straddling both structured and unstructured data, and driving beyond the capabilities of conventional business intelligence and search.”

Stark proof of challenges

My interest was piqued. The scale of the SharePoint challenge is daunting. Consultants, certified Microsoft vendors and Microsoft engineers are working overtime to make SharePoint more useful to its licensees and users. Keeping up with the flood of information about SharePoint is a full-time job. I rely on Alltop.com’s SharePoint headline aggregation service at http://sharepoint.alltop.com and the blog SharePointSemantics.com. However, neither service does an adequate job of keeping pace with Microsoft’s own SharePoint information flow. Essential SharePoint information is scattered across Microsoft’s own site, the MSDN “network” at http://msdn.microsoft.com, Microsoft’s SharePoint team blog, and information services operated by Microsoft experts and dozens of other Microsoft efforts. The flood of Microsoft’s own SharePoint information is stark proof of the challenges SharePoint presents to licensees, developers, consultants and, most important, users.

SBAs are the future

Mounier is an expert in search and all things SharePoint. When asked if other vendors are offering search-based applications as well, he replied, “Of course, but we believe BA Insight pioneered the creation of SBAs on top of the SharePoint 2007 platform, and more recently Fast Search 2010 for SharePoint. Our fastest growing search-based application is called Legal Search for SharePoint and Fast, which provides the attorney with a 360-degree client matter view, securely surfacing documents, e-mails, billing system information and external references such as LexisNexis or West KM in a unified manner. Endeca and Exalead are true pioneers of search-based applications. SBAs are the future, and the market has room for many more innovators. Endeca’s current market focus is on e-commerce and agile business intelligence applications. These are areas we are not targeting.”

After exploring the BA Insight system, the key point of differentiation to me is BA Insight’s leveraging the best-in-class search infrastructure provided by SharePoint and Fast Search, which is now called FS4SP, shorthand for Fast Search for SharePoint. BA Insight does not compete directly with any enterprise search vendors. The net result is that BA Insight can focus on innovating on top of a mature enterprise search platform without having to worry about plumbing or platform extensions such as workflows. It appears that BA Insight’s solution can enhance a SharePoint workflow in the context of the information returned. BA Insight incorporates the collaboration features of SharePoint.

eDiscovery activities can leverage search to dig up more dirt than a Caterpillar convention.  Do you want to know the best data discovery tool that auditors have right now?  It’s your search engine.  You’ve already indexed the content.  All they need is an account and they can find any piece of information that you didn’t want found by just anyone in the organization.  Maybe they’re searching for credit card numbers for a PCI audit, or social security numbers for a PII audit, whatever it is the search tools are going to find it.  And yet, when you want to find something you’re left out in the cold.

Security management is a challenge at every organization.  With every user able to allow other users to see content there are bound to be mistakes.  That’s where you can apply your know how, savvy, and SharePoint techniques to create solutions to help you fight bad security – or bad data practices that can leave you in hot water.

SharePoint search allows you to create search alerts (if enabled).  Search alerts allow you to preconfigure search queries and have the new results that match those queries delivered to you in the form of an email.  These emails then make ready-made lists for documents you should check on each day.  Alternatively, you can use RSS to capture a feed of the items that match the search and have those conveniently archived inside of Microsoft Outlook in a folder for you.  Either way, you’re leveraging search to reach out and let you know when there’s potentially sensitive data being left unsecured.

Of course, you can do the standard searches with your account for things like social security numbers, but in other cases you’ll want to make sure the information isn’t ever stored on the system – like credit card numbers.  For those cases, a special “read all” account can be configured to send alerts to the appropriate parties.

It may be scary to turn on search across all of the repositories in your organization, but security through obscurity isn’t the best policy.  By enabling search you can leverage it to find the problem bits of data lying around your organization – before it gets discovered by the wrong person.