AppletTalk.com Java discussions newsgroups   Archives   FAQ   Search   Memberlist   Usergroups   Register   Profile   Log in to check your private messages   Log in  Native leak on Linux ?          AppletTalk.com Forum Index -> JVM, native methods and hardware View previous topic :: View next topic   Author Message Matthias Ernst Guest Posted: Thu Jul 01, 2004 12:46 pm    Post subject: Native leak on Linux ? Hi, we're having big trouble with our java processes growing and growing in size. It's a servlet application being spidered by httrack and involves heavy filesystem access and CORBA communication. No native code except for the jdk's is involved. Java heap as reported by '-verbosegc' is stable (550-600m used out of -mx768M). No OutOfMemory errors. The number of threads is stable (66). The process as reported by top (e.g. SIZE=1153M, RSS=1.1G) grows slowly but without bounds until it is killed by the OS (within one to a few days). We haven't been able to condense a small testcase. At some point in reducing the problem, it goes away. Systems: j2sdk 1.4.2_04 (tested 1.4.1 as well) redhat 2.1 AS 2.4.9-e.38smp glibc-2.2.4-32.11 (uses /lib/i686/libpthread.so, tried LD_ASSUME_KERNEL=2.2.5 => /lib/libpthread.so as well) same happens on a suse enterprise 8 glibc-2.2.5-165 2.4.19-64GB-SMP Test on Windows and Solaris have been stable. Does anyone have any clues / similar problems / solutions ? TIA Matthias Back to top Jean Charbonneau Guest Posted: Sat Jul 03, 2004 9:55 am    Post subject: Re: Native leak on Linux ? "Matthias Ernst" <matthias.ernst (AT) coremedia (DOT) com> wrote Quote: Hi, we're having big trouble with our java processes growing and growing in size. It's a servlet application being spidered by httrack and involves heavy filesystem access and CORBA communication. No native code except for the jdk's is involved. Java heap as reported by '-verbosegc' is stable (550-600m used out of -mx768M). No OutOfMemory errors. The number of threads is stable (66). The process as reported by top (e.g. SIZE=1153M, RSS=1.1G) grows slowly but without bounds until it is killed by the OS (within one to a few days). We haven't been able to condense a small testcase. At some point in reducing the problem, it goes away. Systems: j2sdk 1.4.2_04 (tested 1.4.1 as well) redhat 2.1 AS 2.4.9-e.38smp glibc-2.2.4-32.11 (uses /lib/i686/libpthread.so, tried LD_ASSUME_KERNEL=2.2.5 => /lib/libpthread.so as well) same happens on a suse enterprise 8 glibc-2.2.5-165 2.4.19-64GB-SMP Test on Windows and Solaris have been stable. Does anyone have any clues / similar problems / solutions ? TIA Matthias Does it grow even without any use of it, or does it grow proportionnaly to the usage ? If it does grow proportionnaly, i'd say there is somewhere a leak which doesn't close connections properly perhaps, have you tried to force gc somewhere ? Sorry I can't help more.. Jean Charbonneau Back to top Roedy Green Guest Posted: Sat Jul 03, 2004 3:12 pm    Post subject: Re: Native leak on Linux ? On Sat, 3 Jul 2004 11:55:48 +0200, "Jean Charbonneau" <kilvaiden (AT) caramail (DOT) com> wrote or quoted : Quote: We haven't been able to condense a small testcase. At some point in reducing the problem, it goes away. see http://mindprod.com/jgloss/packratting.html http://mindprod.com/jgloss/profiler.html -- Canadian Mind Products, Roedy Green. Coaching, problem solving, economical contract programming. See http://mindprod.com/jgloss/jgloss.html for The Java Glossary. Back to top Matthias Ernst Guest Posted: Mon Jul 05, 2004 6:58 am    Post subject: Re: Native leak on Linux ? "Jean Charbonneau" <kilvaiden (AT) caramail (DOT) com> wrote Quote: Does it grow even without any use of it, or does it grow proportionnaly to the usage ? If it does grow proportionnaly, i'd say there is somewhere a leak which doesn't close connections properly perhaps, have you tried to force gc somewhere ? It grows only under load. We cannot see connections leaking --- at least /proc/<pid>/fd/ has no unexpected open files/connections. Is there any way to classify the process' memory from /proc/pid/mem or /proc/pid/maps ? Can anyone read that stuff ? Quote: Sorry I can't help more.. Thanks anyway, we're kind of desperate ... Matthias Back to top Matthias Ernst Guest Posted: Wed Jul 14, 2004 7:01 am    Post subject: Re: Native leak on Linux ? [email]matthias.ernst (AT) coremedia (DOT) com[/email] (Matthias Ernst) wrote in message news:<a71d1113.0407010446.5634dc78 (AT) posting (DOT) google.com>... Quote: Hi, we're having big trouble with our java processes growing and growing in size. To bring this to a conclusion :-) First of all, our measurements were wrong and the growth occurs on all platforms. Second, the culprit is File#deleteOnExit. This call remembers the filename in native memory (no trace of that in Java heap) and does not release the memory when the file in question gets deleted. So this call is to be avoided! There's a corresponding bug report on the JDC. Search for 'deleteOnExit'. Matthias Back to top Inverno Guest Posted: Fri Nov 05, 2004 2:23 pm    Post subject: Re: Native leak on Linux ? Quote: matthias.ernst (AT) coremedia (DOT) com (Matthias Ernst) wrote in message news:<a71d1113.0407010446.5634dc78 (AT) posting (DOT) google.com>... Hi, we're having big trouble with our java processes growing and growing in size. To bring this to a conclusion :-) First of all, our measurements were wrong and the growth occurs on all platforms. Second, the culprit is File#deleteOnExit. This call remembers the filename in native memory (no trace of that in Java heap) and does not release the memory when the file in question gets deleted. So this call is to be avoided! There's a corresponding bug report on the JDC. Search for 'deleteOnExit'. Matthias We've got a pretty similar problem, yet it cannot be related to file management as there's only three files loaded on startup and the rest of data is reached through jdbc. The effective increase of memory allocated happens while using the GUI, it is a growth that can bring the virtual memory allocated to javaw to 300MB+ in a few minutes of use, eventually the application will slow down as the paging gets huge. The application is stable within the 128MB we allocate to it through the usual -Xmx parameter and actually has no troubles working with 80MB while never throwing an OutOfMemoryError, it's the 'external' memory usage that grows, being that a matter between the virtual machine and the OS I can't really see where would be the problem. The leak so far has been detected on two winXP SP1 machines with both the 1.4.1.05 and 1.4.1.04 jre. In the next few days we will test for this leak on two more machines, one of them using winXP SP2. Back to top Display posts from previous: All Posts1 Day7 Days2 Weeks1 Month3 Months6 Months1 Year Oldest FirstNewest First         AppletTalk.com Forum Index -> JVM, native methods and hardware All times are GMT Page 1 of 1   Jump to: Select a forum Java Newsgroups----------------Java DiscussionsWeb-based java games comp.lang.java 3D Graphics API's for Java Java: Support and criticism Java System Announcements comp.lang.java.api JavaBeans Java and CORBA Java and Databases comp.lang.java.developer Java GUI Toolkits Java Help comp.lang.java.javascript JVM, native methods and hardware comp.lang.java.misc Java Language Programming Security and Java comp.lang.java.setup Java Software Tools comp.lang.java.tech ibm.software.java.linux  Non-English Java Newsgroups----------------Java Language (German)Language Java (Français)Java (Italian)  You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum Powered by phpBB © 2001, 2006 phpBB Group SEO toolkit © 2004-2006 webmedic.