Home > Visual Basic Language > Memory Problems - DataTables

Memory Problems - DataTables

I'm getting a major headache using datatables inside VB.NET (Framework 2.0 and 3.5)

I'm basically reading some data from SQL into a datatable and then passing that datatable onto another thread which deals with writing out some data to a COM object before clearing the rows and repeating the loop again.

At the moment the application is reaching over 230Mb+ and then at some point crashing because itran out of memory.

I'm sure that the GCisnt clearing up the datatable or datarows .

The code basically performs the following (is there another way to force the removal of the datarows/datatable from memory)?

--Create array of new records
Dim Ddrows(aMsrTags.Length - 1) As DataRow
For ii = 0 To aMsrTags.Length - 1
Ddrows(ii) = dtDataTable.NewRow()
Next

'Read SQL and copy data into datarows

---Add rows to datatable
dtDataTable.Rows.Add(Ddrows(i - 1))
dtDataTable.AcceptChanges()

--Pass datatable to thread
mythread.DumpDataTable(dtDataTable)
(mergesthe dtDatatable with the thread datatable, maybe its keeping a reference here and not forcing a clean up?)

--remove all rows added
dtDataTable.Rows.Clear()
dtDataTable.AcceptChanges()

--force GC
GC.Collect()
GC.WaitForPendingFinalizers()
GC.Collect()

Repeat process again (this may occur 10000+ times), on average there could be 5000 records in the datatable during each loop.

using Framework V2.0 and even tried V3.5.

Any help please.....

Thanks

Ron

Aussieron Wednesday, September 24, 2008 1:34 AM

Almost all the OOM exceptions I observe so far were caused by improper written application with memory leak or third party components.

Basically, A 32 bit process has 2GB (at most 3GB if OS supports) user mode virtual address space to use. No matter how much physical memory or paging file you have, the 2GB size won't be affected. When the address space is used up (e.g. when the application tries to allocate the memory but cannot find a continuous block of address space large enough to satisfy the allocation request), OutOfMemoryException will be thrown.

OOM exceptions are usually caused by memory leak. For example, the developer allocates memory but forgets to free it after using it. For .Net applications, if we create a large number of objects and keep references to them (like adding event handlers to a WinForm control repeatedly but never removing them), garbage collection will not be able to reclaim them. Hence memory leak occurs.

Occasionally, OOM exception can also be a results of COM call. If the called method return a HRESULT 8007000e (E_OUTOFMEMORY), it can also be translated to OOM exception in .Net.

To diagnostic OOM, we need a sample to reproduce the problem. If that is not possible, we need to collect memory dump (usually at least several hundred MBs each dump) when problem occurs, which is not suitable on forum. I would recommend that you can have a look at the CLR profiler. It shows you how to use the CLR Profiler tool to investigate your application's memory allocation profile. You can use CLR Profiler to identify code that causes memory problems, such as memory leaks and excessive or inefficient garbage collection.

Please remember to mark the replies as answers if they help and unmark them if they provide no help.