perfSONAR User Q&A and Other Discussion

[Andrew Lake <>]

Excellent, thanks! 

Andy


On Feb 18, 2015, at 3:44 PM, Philippe Laurens 
<>
 wrote:

> Hi Andy,
> 
> Yes, I can confirm that the fail.local solution works
> as intended.
> 
> On psmsu01 I installed jail.local, restarted fail2ban,
> cleared the mail queue: zero message have been added
> to the mail queue since.
> 
> psmsu02 was the control sample and just had fail2ban restarted,
> the queue cleared, but the jail files were NOT touched:
> more messages have appeared on its mail queue since this morning.
> 
> Cross-check:  the /var/log/messages file on both nodes
> show ongoing fail2ban activity during that period.
> 
> Thanks,
> 
> Philippe
> 
> 
> On 18-Feb-2015 11:55 AM, Philippe Laurens wrote:
>> Hi Andy,
>> 
>> Ok, thanks.  I can certainly do that, but being sure it truly helped
>> will take some time as, at least for psmsu05, fail2ban was not
>> generating daily emails.  The mailq is still empty since I cleared
>> it yesterday. This may be because this is a machine lesser known
>> to attackers.
>> 
>> Ah! now I see that psmsu01 and 02 are getting similar daily entries
>> in the mailq.  I will install the fix on one of those and we will
>> have a cleaner test.
>> 
>> I am still not sure why the psumxx machines have the same jail.conf,
>> no jail.local and seemingly no messages in mailq.
>> What line is no longer commented out in the fail2ban RPM?
>> 
>> Thanks,
>> 
>> Philippe
>> 
>> 
>> On 18-Feb-2015 10:53 AM, Andrew Lake wrote:
>>> Hi,
>>> 
>>> On the hosts giving you trouble if you copy the attached file to
>>> /etc/fail2ban/jail.local (note that I said copy it to jail.local NOT
>>> jail.conf) and run "/sbin/service fail2ban restart" do the mail issues
>>> subside? It looks like the fail2ban RPM no longer comments out a line
>>> that says to send mail by default. Copying the attached file to the
>>> location specified should override this default. If that helps, we
>>> will squeeze it into the next toolkit release.
>>> 
>>> Thanks,
>>> Andy
>>> 
>>> On Feb 17, 2015, at 5:26 PM, Horst Severini 
>>> <>
>>>  wrote:
>>> 
>>>> Hi Philippe,
>>>> 
>>>> yes, so we're seeing the same problem with fail2ban.
>>>> Not sure why only new installs are affected by that, though,
>>>> and not our production instances.
>>>> 
>>>> The other nightly cron email is a different issue, I think I just
>>>> haven't completely confugred the mesh yet, since I wasn't sure
>>>> whether I should leave that up to the pundit admins to configure.
>>>> 
>>>> So I'm sure that will go away once the mesh is configured.
>>>> 
>>>> Thanks,
>>>> 
>>>>    Horst
>>>> 
>>>> Philippe Laurens 
>>>> <>
>>>>  wrote:
>>>> 
>>>>> Hi Horst et al,
>>>>> 
>>>>> I found a similar status on psmsu05.aglt2.org,
>>>>> but only fail2ban messages are in the queue.
>>>>> 
>>>>> All 164 entries were from Feb 12, 13, 14
>>>>> while psmsu05 had been installed Feb 6-7 (from SL6 and RPMs)
>>>>> 
>>>>> I checked 4-5 of these messages and all those looked like
>>>>> 
>>>>>> [root@psmsu05
>>>>>>  ~]# postcat -q E62D720DC5
>>>>>>    *** ENVELOPE RECORDS deferred/E/E62D720DC5 ***
>>>>>>    message_size:            2420             213
>>>>>> 1               0            2420
>>>>>>    message_arrival_time: Sat Feb 14 11:05:16 2015
>>>>>>    create_time: Sat Feb 14 11:05:16 2015
>>>>>>    named_attribute: log_message_origin=local
>>>>>>    named_attribute: trace_flags=0
>>>>>>    sender:
>>>>>>    original_recipient: 
>>>>>> 
>>>>>>    recipient: 
>>>>>> 
>>>>>>    *** MESSAGE CONTENTS deferred/E/E62D720DC5 ***
>>>>>>    Received: by psmsu05.aglt2.org (Postfix)
>>>>>>            id E62D720DC5; Sat, 14 Feb 2015 11:05:16 -0500 (EST)
>>>>>>    Date: Sat, 14 Feb 2015 11:05:16 -0500 (EST)
>>>>>>    From: 
>>>>>> 
>>>>>>  (Mail Delivery System)
>>>>>>    Subject: Undelivered Mail Returned to Sender
>>>>>>    To: 
>>>>>> 
>>>>>>    Auto-Submitted: auto-replied
>>>>>>    MIME-Version: 1.0
>>>>>>    Content-Type: multipart/report; report-type=delivery-status;
>>>>>>            boundary="7754320DC3.1423929916/psmsu05.aglt2.org"
>>>>>>    Message-Id: 
>>>>>> <>
>>>>>> 
>>>>>>    This is a MIME-encapsulated message.
>>>>>> 
>>>>>>    --7754320DC3.1423929916/psmsu05.aglt2.org
>>>>>>    Content-Description: Notification
>>>>>>    Content-Type: text/plain; charset=us-ascii
>>>>>> 
>>>>>>    This is the mail system at host psmsu05.aglt2.org.
>>>>>> 
>>>>>>    I'm sorry to have to inform you that your message could not
>>>>>>    be delivered to one or more recipients. It's attached below.
>>>>>> 
>>>>>>    For further assistance, please send mail to postmaster.
>>>>>> 
>>>>>>    If you do so, please include this problem report. You can
>>>>>>    delete your own text from the attached returned message.
>>>>>> 
>>>>>>                       The mail system
>>>>>> 
>>>>>>    
>>>>>> <>:
>>>>>>  delivery temporarily suspended: connect to
>>>>>>        example.com[93.184.216.34]:25: Connection timed out
>>>>>> 
>>>>>>    --7754320DC3.1423929916/psmsu05.aglt2.org
>>>>>>    Content-Description: Delivery report
>>>>>>    Content-Type: message/delivery-status
>>>>>> 
>>>>>>    Reporting-MTA: dns; psmsu05.aglt2.org
>>>>>>    X-Postfix-Queue-ID: 7754320DC3
>>>>>>    X-Postfix-Sender: rfc822; 
>>>>>> 
>>>>>>    Arrival-Date: Mon,  9 Feb 2015 10:20:46 -0500 (EST)
>>>>>> 
>>>>>>    Final-Recipient: rfc822; 
>>>>>> 
>>>>>>    Action: failed
>>>>>>    Status: 4.4.1
>>>>>>    Diagnostic-Code: X-Postfix; delivery temporarily suspended:
>>>>>> connect to
>>>>>>        example.com[93.184.216.34]:25: Connection timed out
>>>>>> 
>>>>>>    --7754320DC3.1423929916/psmsu05.aglt2.org
>>>>>>    Content-Description: Undelivered Message
>>>>>>    Content-Type: message/rfc822
>>>>>> 
>>>>>>    Return-Path: 
>>>>>> <>
>>>>>>    Received: by psmsu05.aglt2.org (Postfix, from userid 0)
>>>>>>            id 7754320DC3; Mon,  9 Feb 2015 10:20:46 -0500 (EST)
>>>>>>    Subject: [Fail2Ban] SSH: banned 103.41.124.13 from
>>>>>> psmsu05.aglt2.org
>>>>>>    Date: Mon, 09 Feb 2015 15:20:46 +0000
>>>>>>    From: Fail2Ban 
>>>>>> <>
>>>>>>    To: 
>>>>>> 
>>>>>>    Message-Id: 
>>>>>> <>
>>>>>> 
>>>>>>    Hi,
>>>>>> 
>>>>>>    The IP 103.41.124.13 has just been banned by Fail2Ban after
>>>>>>    5 attempts against SSH.
>>>>>> 
>>>>>> 
>>>>>>    Here is more information about 103.41.124.13:
>>>>>> 
>>>>>>    missing whois program
>>>>>> 
>>>>>>    Regards,
>>>>>> 
>>>>>>    Fail2Ban
>>>>>> 
>>>>>>    --7754320DC3.1423929916/psmsu05.aglt2.org--
>>>>>>    *** HEADER EXTRACTED deferred/E/E62D720DC5 ***
>>>>>>    *** MESSAGE FILE END deferred/E/E62D720DC5 ***
>>>>> 
>>>>> I know very little about fail2ban... I looked around the config
>>>>> files, but could not find where it could be told to send emails.
>>>>> I compared a couple likely candidate config files with another
>>>>> instance (psum05) and could not find any difference either.
>>>>> 
>>>>> 
>>>>> I restarted fail2ban (which put 2 more messages in the queue),
>>>>> cleared the queue, and will be watching.
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Philippe
>>>>> 
>>>>> 
>>>>> On 17-Feb-2015 4:10 PM, Horst Severini wrote:
>>>>>> Hi all,
>>>>>> 
>>>>>> as we have just discussed in the throughput meeting, we are seeing
>>>>>> some strange emails in our newly installed 3.4.1 test machine,
>>>>>> ps3.ochep.ou.edu .
>>>>>> 
>>>>>> One is that fail2ban is now trying to send email to 
>>>>>> ,
>>>>>> claiming to be from 
>>>>>> ,
>>>>>>  so there must be some
>>>>>> incorrect
>>>>>> initial configuration going on during the installation. There's not
>>>>>> a new
>>>>>> version of fail2ban out, though, it's from last August:
>>>>>> 
>>>>>> fail2ban-0.8.14-1.el6.noarch
>>>>>> 
>>>>>> But something else must've changed in the mail configuration
>>>>>> somehow ...
>>>>>> 
>>>>>> The other thing is that the nightly cron job cron-service_watcher,
>>>>>> which runs
>>>>>> /opt/perfsonar_ps/toolkit/scripts/watcher_log_archive_cleanup,
>>>>>> apparently wasn't quite configured correctly, either, since it still
>>>>>> contains
>>>>>> 
>>>>>> STORE_LOCATION=/mnt/store/NPTools/debug
>>>>>> 
>>>>>> which is causing a nightly error mail about that directory not
>>>>>> being there.
>>>>>> 
>>>>>> Did I miss a configuration step there somehow?
>>>>>> 
>>>>>> I installed this machine about a week ago, on one of our old KOI boxes
>>>>>> which used to be our production PS boxes, before we upgraded to the
>>>>>> R310s..
>>>>>> 
>>>>>> Oh, and the other strange thing is that I was trying to install the
>>>>>> other
>>>>>> KOI box, it wouldn't let me, since during the PS installation,
>>>>>> when it comes time to choose where to install the OS on,
>>>>>> anaconda can't find the 160 GB hard drive, so there's no way to
>>>>>> continue
>>>>>> the install.
>>>>>> 
>>>>>> Which makes no sense at all, since I can still boot up the old PS-3.2
>>>>>> instance from the hard drive just fine, so there's obviously
>>>>>> nothing wrong
>>>>>> with the hard drive at all.
>>>>>> 
>>>>>> I also verified all the BIOS settings, they're identical between the
>>>>>> two KOI boxes, so why would one of them allow me to install on the
>>>>>> hard drive,
>>>>>> and the other one won't find it at that time?
>>>>>> 
>>>>>> How can I debug this? We don't immediately need it right now,
>>>>>> but I'd like to be able to install a test bandwidth node there at
>>>>>> some point,
>>>>>> to have a full test set.
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>>    Horst
>>>>>> 
>>>

Attachment: smime.p7s
Description: S/MIME cryptographic signature