Wednesday, October 14, 2009

experiencing issue of out of memory where you run out of permgen space especially
when there is HA Manager DCS events are taking place.
**********Please Note this might be a case by case issue,for exact problem with your case , please contact IBM support by opening a PR using your client account ID***********************

hope this might surely give some insight on the problem
*********************************************************************

CoordinatorCo W HMGR0152W: CPU
Starvation detected. Current thread scheduling delay is 165 seconds.
SystemOut O java.lang.OutOfMemoryError: PermGen space


Apply all the fixpacks available for the Portal server


The following are IBM recommendations:

1# Configure Default Thread Pool size to help DRS traffic. Based on IBM WebSphere Portal 6101
performance tunning guide.
How-To Set: Portal Server->Thread Pools ->DefaultPool=150/150 and disable isGrowable (default=5/20 and isGrowable
is set to true)

2#. Remove the following JVM property based on suggestion from WAS L2 team.

Currently set to true -Dcom.ibm.ws.cache.CacheConfig.propogateInvalidationsNotShared=true
Remove or set it -Dcom.ibm.ws.cache.CacheConfig.propogateInvalidationsNotShared=false


3#. Change the value of the IBM_CS_DATASTACK_MEG custom property to 100MB as suggested by WASCET L2 team.
1. In the administrative console, click Servers > Core groups > Core group settings > core_group_name.
2. Under Additional Properties, click Custom Properties.
3. Change the value specified for the IBM_CS_DATASTACK_MEG custom property.

If the IBM_CS_DATASTACK_MEG property already exists, click on the property name and, in the Value field, specify the maximum amount of
memory that you want to let the core group transport allocate for buffering incoming messages.If this property does not already exist, click New
and then:
a. In the Name field, specify IBM_CS_DATASTACK_MEG.
b. In the Value field, specify the maximum amount of memory you want to let the core group transport allocate for buffering incoming messages.
4. Click OK, and then click Save to save your changes.

4#. Change the size of the Transport Buffer. The maximum amount of heap memory that is specified for the
IBM_CS_DATASTACK_MEG custom property should be less than or equal to the size of the transport buffer.
a. In the administrative console, click Servers > Application servers > server_name, and then, under Additional Properties, click Core
group service.
b. In the Transport buffer size field, specify, in megabytes the size of the transport buffer. Set this to 200MB. Based on IBM
WebSphere Portal 6101 performance tunning guide.
c. Click OK.
d. Repeat the previous three steps for all of the application servers that are members of this core group.
Specify the same transport buffer size for all of these core group members.
e. Click System administration > Node agents > node_agent_name, and then under Additional properties, click Core group service.
f. In the Transport buffer size field, specify, in 200 megabytes the size of the transport buffer.
g. Click OK.
h. Repeat the previous three steps for all of the node agents that are members of this core group.
i. Click System administration > Deployment Manager, and then under Additional properties, click Core group service.
j. In the Transport buffer size field, specify, in 200 megabytes the size of the transport buffer.
k. Click Review to review your changes.
l. Select Synchronize changes with Nodes, and then click Save to save your changes.
Restart all members of the core group.


5# Upgrade Portal to latest fixpack 6.1.0.2.

http://www-01.ibm.com/support/docview.wss?rs=688&uid=swg24022898

6# Apply all the recommended fixes on top of 6102. You can find those fixes on Fix central link.

6.1 Go to fix central link : - http://www-933.ibm.com/support/fixcentral/
6.2 Select WebSphere under Product Group pull down menu.
6.3 Select WebSphere Portal under Product
6.4 Select 6102 Installed Version
6.5 Select All in OS menu.
6.6 Click continue.
6.7 Select Recommended radio button and click continue. This will provide all the recommended ifixes
to be applied on top of 6102 for Portal and Web Content Management 


few more suggestions,


1.  Our asynchronous IO is blocked based on analysis of the thread dumps.  The recommendation is to disable the AIO permanently, using mechanism 1 of the attached link:

        http://www-01.ibm.com/support/docview.wss?rs=180&context=SSEQTP&q1=Disabling%20AIO&uid=swg21366862&loc=en_US&cs=utf-8&lang=en

2.  Set these sizes:

        PermSize:  512 MB
        MaxPermSize:  512 MB
        NewSize:  256 MB
        Min Heap Size:  1796 MB
        Max Heap Size:  1796 MB

3.        We should review the following Portal tuning guide for other settings:

        http://www-01.ibm.com/support/docview.wss?rs=688&context=SSHRKX&context=SS3JLV&context=SS3NNG&context=SSYJ99&context=SSRUWN&context=SS6JVW&q1=Guide&uid=swg27013972&loc=en_US&cs=utf-8&lang=en



No comments:

Post a Comment