Server Memory Woes
Tim Nevels (7/21/08 6:18PM)
Matt Houghton-Thompson (7/22/08 9:24AM)
Tim Nevels (7/22/08 11:00PM)
4D, Inc. (7/22/08 11:27PM)
Tim Nevels (7/21/08 6:18 PM)
On Jul 21, 2008, at 4:35 PM, 4Elias Marmolejo wrote:
Hello, over the weekend we convert our production DB to 11.2 with a
lot of
glitches in the programming mode that has been resolving, but the
part that
I want advise is on the memory side over the past we use to have the
adaptive cache, maximum size 2000 and Minum size 1024, not marked
the keep
the cache and flush every 10 minutes, with this config I have no
problems on
memory with aprox 60 users with a data file of 6 Gb over 4 segments,
this
runs on a Xserve 3Ghz double quad processors with 4 Gb on ram.
On the weekend we have no problems as only log to the DB at most 7
users but
today as all the people start to use the program we get ³Memory
allocation
failed².
I'm not a fan of the "adaptive cache" settings with 4D Server. I like
to set a specific cache memory size based on the server and available
physical memory. And I like to keep the cache in physical memory. So
based on your configuration, I'd try these settings to start:
- Turn OFF adaptive cache
- Set cache memory to 1000MB
- Turn ON "keep the cache in physical memory"
- Flush buffers every 5 minutes
- Turn ON display flush window
One of the configurations you tried was this:
* No adaptive cache and put the size of 300 Mb, keep the cache on
and flush
data every 10 minutes, this gave me no errors over 30 minutes but is
very
slow and also freeze calls and log to the DB, this happens almost
every 5
minutes.
With these setting you did not get the error, but things were "slow".
Keep the cache in physical memory is important, so keep it ON. And
turn OFF adaptive cache so that as soon as 4D Server starts up, it
grabs the whole cache memory, locks it in physical memory, and then 4D
can use the full cache memory as needed.
When you use adaptive cache with 4D v11 Server on Mac OS X, the cache
slowly grows until it reaches the max setting. I'm thinking there is
some problem with this and is causing your error.
Try the above settings and launch 4D Server. Then open the Activity
Monitor and check the "RSIZE" for 4D Server. It should be around
1050MB when 4D Server first launches. Then watch as users log in and
do work. The RSIZE will grow as more user connect. With 60 users
connected it could go to 1500MB or more. Use Activity Monitor in Mac
OS X to see how 4D Server is using memory.
If this memory setting stops the "Memory allocation failed" error
messages, you are going in the right direction. If you are still
having performance issues, you can start increasing the cache memory
size from 1000MB to higher values. Also use the 4D Server window and
all it's memory, network and processor monitoring tools. Watch for
the flush cache window displaying -- see if it is related to your slow
downs. You should be able to track down what is causing your
performance issues then.
Tim
Matt Houghton-Thompson (7/22/08 9:24 AM)
<7cc88da10807220624t1403a259h7ae98347286d7c0a@...
On Mon, Jul 21, 2008 at 7:18 PM, Tim Nevels <timnevels@...
wrote:
With these setting you did not get the error, but things were "slow".
Keep the cache in physical memory is important, so keep it ON. And
turn OFF >adaptive cache so that as soon as 4D Server starts up, it
grabs the whole cache memory, locks it in physical memory, and then 4D
can use the >full cache memory as needed.
When you use adaptive cache with 4D v11 Server on Mac OS X, the cache
slowly
grows until it reaches the max setting. I'm thinking there is some
problem
with this and is causing your error.
This is different than my understanding of adaptive cache, does it
work differently on the Mac? On Windows, it appears to grab the whole
memory on startup.
I noticed the original poster ran out of memory with a cache of 500MB,
that should leave plenty of memory for 60 users. I wonder if there
are one or more client processes taking more memory than expected?
When you modify the cache settings, does 4D report the "Physical
Memory" as 2048 MB?
-Matt H-T.
Sweetwater
Tim Nevels (7/22/08 11:00 PM)
On Jul 22, 2008, at 6:13 PM, Josh Fletcher wrote:
Hi Matt,
You have it right. Cache is always allocated "all at once" and must
be
contiguous.
Kind regards,
Josh Fletcher
Matt Houghton-Thompson wrote:
On Mon, Jul 21, 2008 at 7:18 PM, Tim Nevels
<timnevels@... wrote:
When you use adaptive cache with 4D v11 Server on Mac OS X, the
cache slowly
grows until it reaches the max setting. I'm thinking there is
some problem
with this and is causing your error.
This is different than my understanding of adaptive cache, does it
work differently on the Mac? On Windows, it appears to grab the whole
memory on startup.
I always try to post on the NUG what I discovered during controlled
testing -- what I'm sure about. I want to understand how 4D memory
works, so I conducted some tests. When a 4D Tech Support person
indicates I'm incorrect, it made me question my own testing. So I re-
did those tests again... just to be sure. Tests are run with 4D
Server v11.2 on a MacBook Pro with 2GB RAM running Mac OS X 10.5.4.
Activity Monitor was used to get actual memory allocated to 4D --
"RSIZE".
1. Create a new database with default memory values and then open it
with 4D Server v11.2:
Adaptive Cache ON
Keep cache in physical memory OFF
Max Cache 100MB
Min Cache 32MB
Memory allocated by 4D Server = 75MB
There is plenty of free memory on my system. Why didn't it allocate
the Max Cache of 100MB?
2. Change memory setting to increase cache size and relaunch 4D Server
Adaptive Cache ON
Keep cache in physical memory OFF
Max Cache 400MB
Min Cache 200MB
Memory allocated by 4D Server = 70MB
No change. There is plenty of free memory on my system. Why didn't it
allocate the Max Cache of 400MB, or even the 200MB Min Cache?
3. Change memory setting and and turn off Adaptive Cache and relaunch
4D Server
Adaptive Cache OFF
Keep cache in physical memory OFF
Cache Size 100MB
Memory allocated by 4D Server = 75MB
4D still has not allocated the full 100MB of cache memory specified --
even with Adaptive Cache off. Humm...
4. Change memory setting to keep cache in physical memory ON and
relaunch 4D Server
Adaptive Cache OFF
Keep cache in physical memory ON
Cache Size 100MB
Memory allocated by 4D Server = 138MB
So this time it allocated the full 100MB of cache memory plus 38MB for
4D Server application.
So it seems that the "Adaptive Cache" setting on Mac OS X is a good
description for the setting. Additional testing where you actually
run some methods that create records and fill the cache will show the
memory size of 4D increase up to the cache setting levels -- plus some
application memory for 4D.
The key to getting 4D v11 to allocate the full amount of cache memory
on startup on Mac OS X is to turn ON keep cache in physical memory.
I do not know how memory allocation for v11 works on Windows -- I have
not tested that configuration. It's very possible it acts differently
since Windows uses a different memory allocation system than Mac OS
X. Someone should run these tests with 4D v11.2 on Windows and report
the results. On Mac OS X these test show how it works, but I'm
curious about Windows now.
Tim
4D, Inc. (7/22/08 11:27 PM)
Hi Tim,
I appreciate your thorough testing. I am sorry to be contradictory,
but
I just want to make sure the information is correct. There's a couple
of different topics swimming around here:
Adaptive Cache:
Adaptive cache does not mean the cache size changes when the
application
is running. It means the cache size changes when the machine changes
or, more specifically, when the amount of total memory on the machine
changes. In other words the cache size adapts to the total memory on
the machine. This is useful for software that is deployed on a
variety
of machines, so you don't have to go and change the cache size for
each
machine.
Keep cache in physical memory:
The reason the "RSIZE" varies is just as you say: you're keeping the
cache in *physical memory*, vs. not. Physical memory meaning RAM. As
I'm sure you're aware, modern OS's do not deal only with RAM in terms
of
total memory. In another sense they do not deal with physical memory
*at all*. Virtual memory is used, which is a combination of RAM and
disk space, creating a complete virtual address space.
If the "keep cache in physical" memory option is not checked, then the
OS is free to page out the memory 4D allocated for cache to disk.
This
does *not* affect to total size of the cache. It does *potentially*
offer better performance, assuming the machine has enough free RAM for
4D and other applications.
Check out the 2004 Design Reference on page 158, or the 4D v11 SQL
Design Reference on page 198, it explains this pretty clearly.
Kind regards,
Josh Fletcher
Tim Nevels wrote:
The key to getting 4D v11 to allocate the full amount of cache memory
on
startup on Mac OS X is to turn ON keep cache in physical memory.
Reply to this message
Summary created 7/23/08 at 2:07AM by Intellex Corporation
Comments welcome at: feedback@intellexcorp.com