Reports > Search Logs

Use this page to create and manage search logs, which record user requests for information from your index. You can create and export search logs, then use log analysis software or reporting software to analyze the logs. This page contains the following topics:

• Understanding Search Logs
• About Search Log Format
• About Advanced Search Reports
• Creating Search Logs
• Viewing Search Logs
• Exporting Search Logs
• Updating Search Logs
• Exporting Advanced Search Reports
• Deleting Search Logs
• For More Information

Understanding Search Logs

The search appliance records information about user behavior. This page enables you to create and manage search logs from information recorded by the search appliance. The logs provide the following information about user search requests:

• Type of user queries
• Parts of the user interface users click
• Speed with which results are served
• Whether users are receiving the result they want
• Whether users would find the Related Queries, KeyMatch, Query Expansion, and OneBox features useful

The search appliance stores raw search logs, from which it generates user-requested search log reports. This information is not available through the Admin Console. The search log reports are in Apache web server log format and can be analyzed with any analytical software that accepts Apache web server log format. The raw search log data is maintained for 90 days. After 90 days, the raw data is automatically deleted. You cannot generate search reports for time periods more than 90 days in the past. If you need to analyze log data for longer time periods, generate reports at least once every 90 days, then export and store the reports. You can then analyze the aggregated reports, which were collected over longer time periods. Alternatively, you can set up a syslog server to collect and store raw search log data as it is generated.

You can generate and retain up to 100 search logs reports, which are kept on the search appliance until you delete them using the Delete link. Google recommends that you delete generated reports after exporting them.

Existing search logs are listed at the top of the Search Logs page. You can view, export, update, or delete existing logs. You can also create advanced search reports, which are files in CSV format that contain information about user clicks in the search appliance user interface.

Each log is assigned one of the following status values:

• Complete. The status Complete means that the log is fully generated and ready to be viewed.
• Failed. The status Failed means that an error occurred during generation.
• Regenerating. The status Regenerating means that the appliance is updating the log at user request.
• Generating. The status Generating means that the appliance is generating the log.

About Search Log Format

The search appliance log format is an extension of the Common Log Format (CLF). Each search request is recorded as a separate entry in the log. You can search the logs for entries containing specific strings.

An entry is composed of several tokens separated by spaces:

host -- [date time] request status bytes results time

The following table describes the tokens found in each entry.

About Advanced Search Reports

Each entry in an advanced search report represents a single user click in the search appliance user interface, whether on a search or results page. All strings are URL-encoded. An entry is composed of values for the following items:

• Time of the click to the nearest 100th of a second
• IP address of the search user responsible for the click that was recorded
• Holding place for the session ID, which is always blank
• Click type, which is described in the click type table
• Click start, which is the results page where the user clicked
• Click rank, which is the rank of the result the user clicked
• Click data, which is usually blank
• Query, which is the user query that returned results
• URL of the user click

The following example shows an entry in an advanced search report:

121331555476,172.18.75.121, ,onebox,0,0,,hobo,http%3A//www.tropo.com/

The following table describes search appliance click types.

You can create custom click types for a front end. For more information, refer to "Creating the Search Experience: Best Practices" which is linked to the Google Search Appliance help center.

To enable advanced search reporting, click Enable ASR in the Global Attributes section of the Page Layout Helper. For more information , see Admin Console Help > Search > Search Features > Front Ends > Output Format - Page Layout Helper.

Before Starting These Tasks

Before you create search logs or advanced search reports, ensure that the search appliance has crawled the content and that enough users have made requests that the report is meaningful.

Creating Search Logs

Although the appliance does not prevent you from creating multiple logs at the same time, Google recommends that, for best performance, you create one log at a time.

To create a search log:

1. Navigate on the Admin Console to Reports > Search Logs.
2. On the Show Search Logs for Collection menu, choose the collection whose search queries you want to include.
3. In the Name field, type a report name of up to twenty characters, which must consist of ASCII or non-ASCII characters, hyphens, and underscores. The report name cannot start with a hyphen.
4. Select a Report timeframe and designate the correct date or dates.
5. Click Generate Log. The log is listed in the Report Names column and the Status column displays Generating.
6. To check the status of your report, click the Refresh button on your browser. The page does not automatically update when the report is complete.

Viewing Search Logs

When you view a log, you can search for strings within the log, or jump to specific lines in the log.

To view a search log:

1. Navigate on the Admin Console to Reports > Search Logs.
2. On the Show Search Logs for Collection menu, choose the collection whose logs you want to view.
3. Click the View link corresponding to the log you want to view. The log is displayed.
4. Navigate through the log by clicking the Go to beginning of file, Go to end of file, Previous 50, and Next 50 links.
5. To return to the Search Logs page, Click Back to list of search logs.

Exporting Search Logs

You can export search logs to a file that can be opened with any log analysis software that is able to work with Apache log format. Google recommends that you periodically export your search logs and then delete them from the search appliance. Search logs are saved until you explicitly delete them, and if you use log analysis software on exported, you can combine reports and analyze data generated over a period longer than 90 days.

To generate your own log reports, export the logs in a standard format to a syslog server. (For more information on syslog servers, see Syslog Reports.) You can then run the log analysis software of your choice to customize the reports..

To export a search log:

1. Navigate on the Admin Console to Reports > Search Logs.
2. On the Show Search Logs for Collection menu, choose the collection whose logs you want to export.
3. Click the Export link corresponding to the log you want to export. A File Download wizard opens.
4. To view the file, click Open.
5. To export the file, click Save and navigate to the location where you want to save the file.

Updating Search Logs

You can generate an incremental report before its report period is over, and then update the report to get the final data. For example, suppose you want a report for the month of April. On April 15, you capture the incremental data, and on May 1, you update the report.

While a search log is being updated, its status is Regenerating. You can perform actions on the old version of the report while the new version is being generated.

To update a search log:

1. Navigate on the Admin Console to Reports > Search Logs.
2. On the Show Search Logs for Collection menu, choose the collection whose logs you want to update.
3. Click the Update link corresponding to the log report you want to update. The log report's status changes to Regenerating.
4. To check the status of your report, click the Refresh button on your browser. The page does not automatically update when the report is regenerated.

Exporting Advanced Search Reports

Before you can export an advanced search report, you must create a search log.

To export an Advanced Search Report:

1. Navigate on the Admin Console to Reports > Search Logs.
2. On the Show Search Logs for Collection menu, choose the collection whose logs you want to export.
3. Click the Export ASR link corresponding to the log you want to export. A File Download wizard opens.
4. To view the file, click Open.
5. To export the file as a CSV document, click Save and navigate to the location where you want to save the file.
6. Open the file in a spreadsheet or other analytical program.

Deleting Search Logs

To delete a report:

1. Navigate on the Admin Console to Reports > Search Logs.
2. Choose a collection from the Show Search Logs for Collection drop-down list. The choice All Collections displays only those logs that apply to all collections, not all logs for each individual collection.
3. Click the Delete link corresponding to the log you want to remove. The search appliance displays a dialog box asking you to confim the delete.
4. Click OK. The log is removed.

For More Information

For more information about front ends and collections, see "Creating the User Experience," which is linked to the Google Search Appliance help center.