Developer forum

Forum » CMS - Standard features » New Index and very large table

New Index and very large table

Adrian Ursu
Reply

Hi Guys,

I am trying to use the new index for creating an SQL index using 2 tables. One of the tables has about 9 milion rows and we expect it to increase considerably over time.

I have tried to run the index but so far it times out. Is there any way I can increase the timeout setting?

Is it even possible to accomplish this indexing using the New Index with Lucene?

Thanks,

Adrian


Replies

 
Nicolai Høeg Pedersen
Reply

Hi Adrian

What times out? The SQL-server, the IIS or Lucene?

 
Adrian Ursu
Reply

It returns a red message when I try to Build the index. Not sure which one of the three is the cause.

Thanks,

Adrian

 
Nicolai Høeg Pedersen
Reply

And waht does that message say?

 
Adrian Ursu
Reply

Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

After this, I cannot try and run it again because the index gets locked

Lock obtain timed out: NativeFSLock@C:\Domains\Sites\floydpepper.dw-demo.com\Files\System\Indexes\ProductswSerialNumbers\ProductswSerialNumbers\A\write.lock

I will try first to optimize my SQL statement and maybe add some indexes on the large table.

Thanks,

Adrian

 

 

 
Nicolai Høeg Pedersen
Reply

That is a SQL-server timeout - so you need to find a way to make the SQL faster. It is indexing using a reader, so it should be possible to go through all records. But it is a BIG job indexing 9 mio records, so you need to make sure you have the right server setup so SQL + IIS has enough memory and does not restart in the middle of the process.

 
Adrian Ursu
Reply

Understood. 

I have tried indexing just the big table, without any joins and it runs ok in about 3 minutes for 9 milion records.

I am not sure if I can expect it to run in 53 minutes on the final 160 milion rows :)

Thanks,
Adrian

 

You must be logged in to post in the forum