Troubleshooting Crawl Issues for SSL Enabled Web Application

Lately I have experienced lot of issues while crawling of SSL enabled web application and wasted almost 2-3 days fixing it. So thought to share the troubleshooting steps on my blog.

Issue 1 : Ignore SSL Warnings

Error message: An unrecognized HTTP status was received. Check that the address can be accessed using Internet Explorer. (WebExceptionStatus TrustFailure The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

This happens mostly when we implement self signed certs or certs from untrusted authorities in the web servers. In order to avoid the error message and to successfully crawl contents, make  the following changes in farm level search settings in Central Administration.

1. Go to Application Management > Search Service Application Settings > Farm Search Administration
2. Change the setting “Ignore SSL warnings” from No to Yes

Issue 2 : Install the FAST search Certificate

The URL of the item could not be resolved. The repository might be unavailable, or the crawler proxy settings are not configured. To configure the crawler proxy settings, use Search Administration page.

Install the certificate using the following command on the Sharepoint Admin Server
.\SecureFASTSearchConnector.ps1 –certPath FASTSearchCert.pfx –ssaName “FAST Connector” –username "domain/username"

Note : The FASTSearchCert.pfx is created when you install the FAST on the server. You can locate it under <FASTSEARCH>\Data\Data_Security\cert folder. Also on the execution of the command it will ask for the certificate password, It is the password that you entered while configuring the FAST on the server.

Issue 3 : Collection not created

Failed to connect to ame.fantasia.qa:13391,ame.fantasia.qa:13391 Failed to create session with content distributor. Unknown collection 'MyCollectionTest'

This happens when collection specified the SSA is not present in the FAST database. To check if your collection exist you can execute the following command on the FAST server. 

                 Get-FASTSearchContentCollection -Name MyCollectionTest
or else you can create a new collection with the following command 
                  New-FASTSearchContentCollection -Name MyCollectionTest

For me it was not present hence gave me an error. For quick fix, i changed the collection name back to sp. Its the default collection and was already present.

Issue 4 : Access Denied to the Default Content Access Account

Access is denied. Verify that either the Default Content Access Account has access to this repository, or add a crawl rule to crawl this repository. If the repository being crawled is a SharePoint repository, verify that the account you are using has "Full Read" permissions on the SharePoint Web Application being crawled. 

This happens when the content access account does not have "Full Read" permissions on the Sharepoint web application being crawled. You can add a crawl rule to access the repository with specific account.

1. Go to New Crawl Rule -> add the Url 
2. Check the radio button for Specify a different content access account
3. Enter the credentials for the account with which you want to crawl.
4. Click Ok and start the crawl.

Also make sure there is an entry for DisableLoopbackCheck in the registry. 
If its not present follow the below steps :

1. Go to command window and type regedit.exe
2. Navigate to  HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
3. Right click on LSA and create a new 32bit DWORD value
4. Name it as "DisableLoopBackCheck" and modify its value to 1
5. Close the registry and crawl the content.

 Issue 5 : Error communicating to the server

The crawler could not communicated with the server. Check that the server is available and that the firewall access is configured correctly. If the repository was temporarily unavailable, an incremental crawl will fix this error.

This happens when crawler is not able to connect to the server. Make sure server name is correct. Couple of steps to troubleshoot it

1. You should be able to ping the server from the server having crawl component. Make sure there is an entry for the server in the host file under c:\Windows\System32\drivers\etc folder.
    Ping <servername>
2. You should be able to connect to the server using telnet command
   Telnet <servername> <port number>