#include <StringConstants.au3> ; to declare the Constants of StringRegExp
#include <Array.au3> ; UDF needed for _ArrayDisplay and _ArrayConcatenate
Local $sRegex = "(?m)best practices "
Local $sString = "BEST PRACTICES – CODING STANDARDS" & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"AUTOSYS:" & @CRLF & _
"" & @CRLF & _
"1. NRETRY OPTION Feature - Extract jobs querying against databases should use Autosys N-retry option 3-4 times before failure." & @CRLF & _
"2. Health Check job/Dummy Job/Polling Job mechanism should be designed to have better operations controls (Holding, Releasing Jobs)" & @CRLF & _
"3. The server names should not be hardcoded in Autosys JILs. Wide IP/ Alias/Virtual Machine names should be used." & @CRLF & _
"4. Multifilewatcher concept should be pursued instead of file watcher to reduce bulk failures during planned outage." & @CRLF & _
"5. Separate PROD and COB Schedules - Maintain separate Autosys jobs for PROD and COB loads, wherever DUAL LOAD mechanism exists. This will help us to HOLD/Release all the Jobs, which are connecting to a particular DB/Server." & @CRLF & _
"6. Restrict # of multiple unloads from source DB using Autosys feature of controlling # of parallel jobs. Have a health check job where we are having a scenario of multiple unloads from a Database." & @CRLF & _
"7. Resource concept should be used in Autosys to achieve load balancing and avoid performance bottlenecks due to high number of parallel executions." & @CRLF & _
"" & @CRLF & _
"8. Job Naming Convention- Job Name should specify as much info as possible. Eg,1. A short code for the project/application name, 2. Type of Job (Extract, Transformation, Load, File Watcher etc.), 3. Infra getting used (DB Name alias) " & @CRLF & _
"" & @CRLF & _
"9. The Autosys JILs should have a comment describing the brief function of the job. " & @CRLF & _
" Eg: description: "This job is to unload the data from XYZ EDW table and to load ABC folder/system"" & @CRLF & _
" Description: "This job is validates the source file and generates load ready file"" & @CRLF & _
"" & @CRLF & _
"10. There should not be any jobs without a box unless it is a standalone job. " & @CRLF & _
"" & @CRLF & _
"11. Schedules should have Time dependency as well as trigger dependency (If Source is file or Source DB have trigger mechanism to mark the completion of loading of data)" & @CRLF & _
"12. Lookback feature should be added to avoid accidental execution of schedule by referring to the success state of previous schedule." & @CRLF & _
"13. Jobs to be put on schedule only when the project is planned to be LIVE." & @CRLF & _
"14. Use bulk and Intermediate mode concept based on data volume" & @CRLF & _
"15. Applications should not be scheduled during Green Zone " & @CRLF & _
"16. Projects that are obsolete in prod needs to have jobs decommissioned instead of being ON-ICE" & @CRLF & _
"17. Deleting Jobs - While Jobs are being decommissioned, proper dependency analysis should be done. The Box should be deleted if all the Jobs in that box is to be decommissioned." & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"TRIGGER FILES:" & @CRLF & _
"" & @CRLF & _
"1. All the triggers and source files should be cleaned up/archived after processing. " & @CRLF & _
"2. Files being sent to BI should have data files and trigger file to ensure downstream receives the data file and processes it. Eg: ODS sends files to BI, each data file is accompanied with trigger file." & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"CLEANUP & UTILITY SCRIPTS:" & @CRLF & _
"" & @CRLF & _
"File Retention Policies/Cleanup/Space Monitoring – " & @CRLF & _
"1. Define retention policy for every file – Input, output, intermediate, reject, error files etc." & @CRLF & _
"2. Ensure the daily cleanup process is setup for every feed in production and new feeds being promoted to production based on retention policy." & @CRLF & _
"3. Ensure that all new projects have space allocation done to all the required mounts / directories based on retention policy before implementation of the project to production and ensure space-monitoring scripts are available." & @CRLF & _
"4. Log file locations and not coding to have any files in /tmp or any other non-project locations." & @CRLF & _
"5. Auto cleanup of files including inbound, outbound, error, log, Autosys logs" & @CRLF & _
"6. Automated reconciliation controls in place to catch issues in data so that to avoid any manual work around and fix issues in a timely manner." & @CRLF & _
"7. Reject handling frameworks- Rejects should be captured and sent to downstream/ users" & @CRLF & _
"8. " & @CRLF & _
"" & @CRLF & _
"FILE HANDLING:" & @CRLF & _
"" & @CRLF & _
"1. Header / trailers should be encouraged to assist reconciliation process and help in audit/controls .Ex: Provide record /data count in file." & @CRLF & _
"2. Source file having transactional Data with Dates : A sanity check should be done to ensure that duplicate data is not being processed" & @CRLF & _
"3. Put controls to ensure the Files containing PII information is contained in a special directory which is not accessible and secure (Not use persistent access, but use a specialized FID)" & @CRLF & _
"" & @CRLF & _
"AUDIT & CONTROLS:" & @CRLF & _
"1. NDM Jobs – Transmission controls to ensure all data has been transferred successfully" & @CRLF & _
"2. FID - Each Application ( Example: Global Fraud, REW, Genesis) within BI should have their own Unique FID’s for batch processing" & @CRLF & _
"3. Alerts – Only actionable alerts should be set up for Production Management. " & @CRLF & _
"4. Access permissions - As of now all Codes, parm, config, env are in FID, which is high risk. Code and environment parameters should be owned by an FID which would be accessible by only SA/Admin" & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
"EAP –" & @CRLF & _
"" & @CRLF & _
"1. Edge node - Identify Key Players for EAP in terms of which team dictates App to use Edge Nodes, team responsible to allocate maximum space for a give App etc. Currently there are 3 Edge Nodes and all new projects are using same Edge Node - bdswr402x20d1.nam.nsroot.net. " & @CRLF & _
"2. Include Kerberos Authentication script in jobs to ensure continuity of job run else it will time out/terminate after 8hrs. Kerberos Authentication is at FID level so if multiple teams are using same FID, one job in any of these teams should take care of the authentication" & @CRLF & _
"3. CATE framework needs to be flexible to provide custom paths for applications to write their log/error files etc. For debugging purpose. Currently all Talend logs are dumped into this location /var/log/talend/." & @CRLF & _
"4. All the Autosys logs should be captured in application specific edge node location. Admin should have space limit set to how much each FID can write on /home directory." & @CRLF & _
"a. As per current design , Autosys logs are being captured in /home " & @CRLF & _
"" & @CRLF & _
" std_out_file: "/home/gcgeeapnatldp/autosyslogs/$AUTO_JOB_NAME.$AUTORUN.out"" & @CRLF & _
"std_err_file: "/home/gcgeeapnatldp/autosyslogs/$AUTO_JOB_NAME.$AUTORUN.err"" & @CRLF & _
"" & @CRLF & _
"" & @CRLF & _
""
Local $aArray = StringRegExp($sString, $sRegex, $STR_REGEXPARRAYGLOBALFULLMATCH)
Local $aFullArray[0]
For $i = 0 To UBound($aArray) -1
_ArrayConcatenate($aFullArray, $aArray[$i])
Next
$aArray = $aFullArray
; Present the entire match result
_ArrayDisplay($aArray, "Result")
Please keep in mind that these code samples are automatically generated and are not guaranteed to work. If you find any syntax errors, feel free to submit a bug report. For a full regex reference for AutoIt, please visit: https://www.autoitscript.com/autoit3/docs/functions/StringRegExp.htm