Fork me on GitHub
 

Frequently Asked Questions for Searchdaimon Es

Table of Contents

Common tasks

Searching without logging in (anonymous user)

General

What is Enterprise Search?

What can Enterprise Search do?

Why choose Searchdaimon?

What types of documents can I search?

What types of media can I search?

Search is slow

VM slow for first response after some time idle

It take a long time to login to see the search box. Then Search is fast

System

ES reports “Couldn’t contact the search system. Please try again later.”

Is there a maximum file size the ES can handle / crawl / index?

Updating the ES from behind a Proxy Server

Crawling

Turn off indexing of certain web pages

Geting a “Out of disk space” but have some free space

What is the robot name of the Intranet crawler?

User experience

How do I change the search interfac language?

Why do I sometime get more/fewer hits when I change language?

Outlook reports “The operation failed” when opening link to Public Folder

Can’t open links in results (links with file://, outlook: etc)

VMWare specific

Mounting local filesystems: Error cannot mount filesystem: Protocol error

VirtualBox specific

Unable to boot: This kernel requires an x86-64 CPU, but only detected an i686 CPU.

Windows

How do I open e-mail directly from my browser?

ES on you own server

Install error: Specified nonexisting disk sda in clearpart command

Development

Running Perl from the command line




Common tasks

This is an overview over popular tasks in Searchdaimon ES.

Searching without logging in (anonymous user)


Searching without logging in (anonymous user)

When you want documents to be available to the public, you can enable the anonymous user. This way you don't have to log in to access the search interface.

In the administration panel, go to Overview. Select "Manage" on the collection you want to be publicly available. Go to the "Advanced management" tab, and scroll down to the "Edit settings for Administration" section. Under "Anonymous collection", select "Enable anonymous search", and click "Submit changes".

The changes will take effect immediately. To access the public interface, add "/public" to your url: http://<ip-address>/public

General

As a bare minimum, you have to do the following tasks before you can start using Searchdaimon ES.

What is Enterprise Search?

What can Enterprise Search do?

Why choose Searchdaimon?

What types of documents can I search?

What types of media can I search?


What is Enterprise Search?

Enterprise Search enables you to search content throughout your company. This includes documents, e-mails, databases and more.

What can Enterprise Search do?

You can save time looking for your lost documents. Or, if you are working on a new topic, you can search to see if anyone in your company has done a related task before.

Why choose Searchdaimon?

Searchdaimon ES is scalable, fast, and easy to configure and use. It also comes at more affordable prices than other solutions.

What types of documents can I search?

Searchdaimon ES supports most popular formats. This includes Microsoft Office, OpenOffice and Adobe Documents.

What types of media can I search?

We support Windows shares, intranet web, Exchange, SuperOffice, Sharepoint and more. You can even write your own connector.

Search is slow

This is almost always caused by a couple of known problems. Ether slow on-line validation of privileges, ES processes being swapped out of memory or slow response from your user system when validating credentials.

VM slow for first response after some time idle

It take a long time to login to see the search box. Then Search is fast


VM slow for first response after some time idle

If you use the virtual appliance version of Searchdaimon ES, it can also be caused by VMware. Some times this only happens the first time you search. If this is the case, then most proboly the ES's memory is beeing swaped out by the hypervisor. Try setting a memory reservation for the virtual appliance to avoid this from happening

Performance can be improved by removing other virtual machines from VMware, or by installing more memory on the VMware server. A hardware installation of Searchdaimon ES doesn't have these issues.

It take a long time to login to see the search box. Then Search is fast

When you log on you are prompted for username and password. ES verifies thus credentials with your user system (Active Directory or other ldap system). ES also need to get witch group a user is member of. If your user system is slow to respond, login can take some time.

System

ES reports “Couldn’t contact the search system. Please try again later.”

Is there a maximum file size the ES can handle / crawl / index?

Updating the ES from behind a Proxy Server


ES reports “Couldn’t contact the search system. Please try again later.”

When you try to search, you get the following error message: "Couldn't contact the search system. Please try again later."
This can be caused by several things. Wait some seconds and try again. If the error is still there, open the Administration panel, go to "Manage services", and check if all services is labeled "is running...". If any are stopped, click "Restart". You can also try to restart "Crawling Management" and "Search Service". Wait a couple of minutes, and try a new search. It might take a while for the system to respond the first time. If the error doesn't disappear, contact us.

Tip: All logs are available at "Statistics & Logs" -> "Log files".

Is there a maximum file size the ES can handle / crawl / index?

The ES index all data in files up to 100mb in size. If the file is larger than that, only the file name will be indexed.

Updating the ES from behind a Proxy Server

Unfortunately setting up a proxy server for automatic updates isn't supported from the administrator interface yet. But it can be done manually by editing a configuration file.

 

First log in as root from the console.

You most then edit in /etc/yum.conf to specify the proxy server details. The proxy setting must specify the proxy server as a complete URL, including the TCP port number and a trailing /. If your proxy server requires a username and password, specify these by adding proxy_username and proxy_password settings.


The settings below enable yum to use the proxy server mycache.mydomain.com, connecting to port 3128, with the username yum-user and the password qwerty.

 

# The proxy server - proxy server:port number 
proxy=http://mycache.mydomain.com:3128/ 
# The account details for yum connections 
proxy_username=yum-user 
proxy_password=qwerty

 

Remember the "/" at the end!

You should then be able to use Software updates->Software updates->Check for updates from the administrator interface as normal.

Crawling

Turn off indexing of certain web pages

Geting a “Out of disk space” but have some free space

What is the robot name of the Intranet crawler?


Turn off indexing of certain web pages


If you don't want some of your web pages to be indexed, you can tell our crawler by appending user-agent sdbot-robot to your robots.txt, as described in The Robots Exclusion Protocol:

    User-agent: sdbot
    Disallow: /folder/

Geting a “Out of disk space” but have some free space

In addition to storing crawled data the ES need room for log files and temporary data during indexing. For this reasons crawling will stop with a "Out of disk space" error if you have less then 4 GB of free disk space on partition /boithoData.

Please see the Increasing the size of the virtual disk in VMware section for example on how to increase a virtual disk.

What is the robot name of the Intranet crawler?

It is "sdbot".

Correct syntacs to allow the crawler to index "folder".

    User-agent: sdbot
    Allow: /folder/

User experience

How do I change the search interfac language?

Why do I sometime get more/fewer hits when I change language?

Outlook reports “The operation failed” when opening link to Public Folder

Can’t open links in results (links with file://, outlook: etc)


How do I change the search interfac language?

The search interface is shown in the browser's standard language. To add a language in Internet Explorer: Choose "Tools" -> "Internet Options", click "Languages", click "Add" and pick your language from the list. If the browser is configured with more than one language, the top-most one will be used.

At the time being, Searchdaimon ES supports the English and Norwegian languages.

Why do I sometime get more/fewer hits when I change language?

The search engine use linguistic knowledge to reduce senses of words to their common stem. When searching for "technology", you'll also get documents containing the word "technologies". But only if your search interface runs in english. If you search for words in another language than that of the search interface, this linguistic knowledge will disappear. This also holds true for synonyms, if you have installed a package of synonyms.

Tip: If you encapsulate words in parantheses, stemming and synonyms will be turned off.

Outlook reports “The operation failed” when opening link to Public Folder

When you try to click on a link to an Exchange Public Folder in the search interface, Outlook freezes and reports: "The operation failed."

This can happen if Outlook hasn't been syncronized with the public folder for a while. Open Outlook and click on the Public Folder. Outlook should now syncronize.

Can’t open links in results (links with file://, outlook: etc)

Different browsers block access to local files for security reasons. Please se End-user integration, local client software and patching for suggested workarounds.

VMWare specific

Mounting local filesystems: Error cannot mount filesystem: Protocol error


Mounting local filesystems: Error cannot mount filesystem: Protocol error

During boot you may get the message "Mounting local filesystems: Error cannot mount filesystem: Protocol error" as shown below:

This message is harmless and can safely bee ignored. It simply means that you are not sharing files between the virtual machine and the host using HGFS.

You may read more about HGFS at http://www.vmware.com/support/ws4/doc/running_sharefold_ws.html.

VirtualBox specific

Unable to boot: This kernel requires an x86-64 CPU, but only detected an i686 CPU.


Unable to boot: This kernel requires an x86-64 CPU, but only detected an i686 CPU.

When starting the ES you get the boot error "This kernel requires an x86-64 CPU, but only detected an i686 CPU. Unable to boot – please use a kernel appropriate for your CPU".

This error message indicates that there is an issue with 32 / 64 bit emulation. The Searchdaimon ES uses a 64 bit operating system. VirtualBox should, however, be able to handle this even on a 32 bit host.


If you have a 32 bit cpu which supports hardware virtualization this must be enable. Please access your machines bios and see if you can enable it (normally it is called Intel Virtualization Technology (VT-x) or AMD V). If you don’t have the option to enable hardware virtualization you will probably not be able to run the ES.

Further information on this issue is available in this excellent blog post.

Windows

How do I open e-mail directly from my browser?


How do I open e-mail directly from my browser?

You may need a special register key to allow this. Please see Exchange with Outlook 2008 or 2010 in Windows in the End-user integration, local client software and patching.

ES on you own server

Install error: Specified nonexisting disk sda in clearpart command


Install error: Specified nonexisting disk sda in clearpart command

During instalation you get the error below.

Kickstart sda error

This is noramly casued by the harddisk configuration. Some bios ships with a fake-raid setup that require one to load a proprietary driver during the installation. Fake-raid is generelt bad, and should normally only be used for PC's where you want to dual boot Linux and Windows. The best is just to disable this function all together. Reboot and enter the bios, then set SATA mode selection to "Native Mode".

Where in the bios this option is located may very from manufacturer to manufacturer. For HP it is lokated at Advanced -> Harddisk Configuration.

Sata mode

Development

Running Perl from the command line


Running Perl from the command line

The embedded version the ES is using is 32 bit version of Perl. But /usr/bin/perl is 64 bit. You may therefor experiences some differences in available CPAN modules if you develop scripts from the commandline, then later embed them in the crawler or usersystem. To run Perl with the same environment and with the same modules as the ES , use the /home/boitho/boithoTools/bin/perl binary.

 

Copyright © Searchdaimon AS. All rights reserved.