QRoC SIEM integration scripts
CyberHat publishes codes on open-source platform Github to improve global security
Within the cyber security community, IBM’s QRoC software is something of a catch-all when it comes to managing security information. QRoC (like other SIEM technologies) provides the ability to take information from multiple security tools and create rules that allow the data to be managed in a single space, analyzing them for correlations in real time.
For this strategy to be effective, however, the security products used by the organization must be able to synchronize with IBM QRoC and forward the relevant logs. While IBM QRoC has made sure firewalls, antivirus software, security products, servers and databases are able to be integrated within their technology, there are still many tools that are falling through the cracks, which can affect an organization’s ability to properly secure its network.
The Issues: Correlation and Visibility
When a company invests in cyber defense tools, it is making those purchases with the assumption that these platforms will address pressing cyber-security needs. But for that to be true, those tools must be configured and integrated properly. This means synchronizing, optimizing, and properly managing all platforms. Ideally, this would be done by a knowledgeable team that sees the entire network topology, understands the security needs and is able to align them with the needs of the business.
We developed unique script for QRoC with Mimecast.
We are now releasing those script to the public, allowing the cyber and IT security community to utilize them through step-by-step installation guides.
Step 1 - Running Mimecast python script - (the script return mimecast log files):
- Copy the script into the EC as python file. (as connectorMimecast.py)
- Insert the relevant parameters for the Authentication: ("Set up variables") lines 15 -23 in the script:
APP_ID = "YOUR DEVELOPER APPLICATION ID"
APP_KEY = "YOUR DEVELOPER APPLICATION KEY"
URI = "/api/audit/get-siem-logs"
EMAIL_ADDRESS = 'EMAIL ADDRESS OF YOUR ADMINISTRATOR'
ACCESS_KEY = 'ACCESS KEY FOR YOUR ADMINISTRATOR'
SECRET_KEY = 'SECRET KEY FOR YOUR ADMINISTRATOR'
LOG_FILE_PATH = "FULLY QUALIFIED PATH TO FOLDER TO WRITE LOGS"
CHK_POINT_DIR = 'FULLY QUALIFIED PATH TO FOLDER TO WRITE PAGE TOKEN'
Step 2 - Prepare environment on Linux:
- Create the Natives:
- mkdir /root/mimecast/
- mkdir /root/mimecast/forwardlogs/
- mkdir /root/mimecast/forwardlogs/logs_befor_csv
- mkdir /root/mimecast/forwardlogs/logs_csv
- mkdir /root/mimecast/forwardlogs/logs_delete
- Make sure that the following packages are installed:
- logging, logging.handlers,json,os, requests, base64, uuid, datetime,hashlib, hmac, time
Step 3 - Run Linux command to schedule and change the format log to support with QRadar using Crontab:
- Login to EC by SSH (as root)
- insert to Red-Hat linux scheduler:
- crontab -e
- Type i to start typing.
- Insert the next two commands:
The command converts the logs files from mimecast cloud to QRoC readable csv file format, sending the logs to QRoC Console by syslog and delete all the used log files.
Note: <ClientIdentifiyer-Mimecast> is the identifyer filed for QRoC, please replace it by the relevant format.
- Example: "clientsname"-Mimecast (Please find the yellow mark at the Example bellow)
*/1 * * * * head -800 /root/mimecast/logs/* > /root/mimecast/forwardlogs/logs_befor_csv/logs.txt;sed 's/,//g' /root/mimecast/forwardlogs/logs_befor_csv/logs.txt > /root/mimecast/forwardlogs/logs_befor_csv/logsNoComma.txt;cat /root/mimecast/forwardlogs/logs_befor_csv/logsNoComma.txt > /root/mimecast/forwardlogs/logs_csv/logsReady.csv;awk '$1 != "==>"' /root/mimecast/forwardlogs/logs_csv/logsReady.csv >> /root/mimecast/forwardlogs/logs_csv/output.csv;cat /root/mimecast/forwardlogs/logs_csv/output.csv > /root/mimecast/forwardlogs/logs_befor_csv/logs.txt;sort -k1 -r /root/mimecast/forwardlogs/logs_befor_csv/logs.txt >/root/mimecast/forwardlogs/logs_befor_csv/logsNoComma.txt;sort -k1 -r /root/mimecast/forwardlogs/logs_befor_csv/logsNoComma.txt > /root/mimecast/forwardlogs/logs_befor_csv/logs.txt;cat /root/mimecast/forwardlogs/logs_befor_csv/logs.txt > /root/mimecast/forwardlogs/logs_csv/logsReady.csv;sort -k1 -r /root/mimecast/forwardlogs/logs_csv/logsReady.csv > /root/mimecast/forwardlogs/logs_csv/output.csv;/opt/qradar/bin/logrun.pl -f /root/mimecast/forwardlogs/logs_csv/output.csv -u <ClientIdentifiyer-Mimecast> 1000;rm -rf /root/mimecast/forwardlogs/logs_csv/*;rm -rf /root/mimecast/forwardlogs/logs_befor_csv/*;cd /root/mimecast/logs/ &&(mv `ls | head -800` /root/mimecast/forwardlogs/logs_delete);rm -rf /root/mimecast/forwardlogs/logs_delete/*;
The command executes connectorMimecast.py python script that collect the logs from mimecast cloud by REST API every 3 months.
- * * * */3 * python /root/mimecast/connectorMimecast.pyType Esc :wq! to save the actions.