Shibboleth Developers

[André Cruz <>]

On 2008/01/02, at 18:16, Scott Cantor wrote:

> > I used test-client.. Ran it 1000+ times and the memory graph is  
> > pretty
> > similar, albeit with a much smaller time frame:
>
> Meaning the memory usage is the same? That's surprising to me, I  
> admit.
>
> But I think to measure this, you would need to avoid just spinning  
> up 1000
> copies of the client, which would each connect to a socket. That's  
> not how
> the SP is intended to work...it favors multithreaded web servers, and
> tolerates prefork provided you keep the process count sane.

I hacked test-client to loop 100000 times and ran 4 processes  
simultaneously. The memory graph didn't climb this time so it's really  
a thread/socket-related problem.

> There shouldn't be 1000 sockets open in a real deployment, and if  
> there is,
> that's probably the real problem, not the cleanup code.

Not at the same time no... But if the threads don't clean up their act  
when they finish then eventually problems will occur.

> Secondly, you have to take into account what I'm saying...Linux will  
> NOT
> reclaim all that memory. It just won't. I can't fix that. What it  
> will do is
> reuse it. So if you drive the process to 1G and then let all the  
> sockets
> close when the client processes shut down, and then do it again,  
> what is the
> delta?

Ok... For this test I used the original test-client. I ran it a couple  
of hundred times and the memory build up.. Then I let it rest for 10  
mins... Then started it up again. As you can see the memory is not  
reused. If this was the case than I would have no problem with it. If  
it used 1GB constantly than it would be strange, but ok. What happens  
is that all of the memory is used if the process is not stopped.

I'll look into the RPC cleanup code tomorrow if I get the chance...  
It's the best candidate. Unless there is some other test to help solve  
this?

Regards,
André