In the context of incident response, “malware” analysis plays a critical role, as it is a capability that can be key to the resolution and full understanding of its impact. After detection of a malicious artifact, understanding how it works and its impact on the environment can be critical to containing the incident and preventing further damage. In addition, it can help to understand the adversaries’ behavior and objectives and, of course, help the organization to prevent future similar attacks. However, since it is a complex skill in many respects, often analysts may not possess advanced skills in programming or in the use of debuggers/disassemblers and, because of this, may feel unable to perform a useful analysis, almost always limiting knowledge about a specific malicious artifact to automated analyses performed in online sandboxes such as VirusTotal or AnyRun. The purpose of this article is to clarify that limited knowledge, at first, should not be considered a barrier to performing a good analysis.
The first step is to understand that there are several ways to scan a suspicious file. While not an exhaustive list, the following are the most common types of scans:
- Fully automated analysis in sandboxes (online and/or offline).
- Analysis of the static properties of the binary (e.g., Strings and Imports / API Calls).
- Interactive analysis of binary behavior.
- Manual code analysis / Reverse code engineering.
By combining behavioral analysis techniques and static properties, it is possible to identify new indicators of compromise (IoCs) that are not discovered in fully automated analysis, trace the impact of malware, and plan rapid and strategic responses. However, this does not change the fact that, in some situations, these techniques must be complemented with more advanced steps, such as code analysis and reverse engineering of the binary.
It should be noted that there is an inversely proportional relationship between the quantity/quality of the results obtained, the time required for the analysis and the difficulty of the process. Lenny Zeltser, in a publication for SANS in 2021, organizes the types of analysis in a pyramid, positioning them in increasing order of complexity, from the base to the top, where manual code analysis and reverse engineering of the code is found.
To conclude this introduction, it is important to note the following: in a part of the real cases of incidents involving malware, fully automated analysis, supported by a good OSINT, will be sufficient to discover the capabilities of the artifact, since, most likely, your organization will not be the first to face such a sample. On the other hand, in slightly more complex situations, behavioral analysis combined with static property analysis can provide very conclusive results.
In other words, the most advanced analysis, which Zeltser places at the top of the pyramid, is generally applied in highly complex situations, with artifacts designed specifically for your organization and which usually require a certain degree of confidentiality in the investigation, or in cases where simpler analyses do not produce the expected results. The idea is to always move from the simplest to the most complex in order to avoid duplicate work and take advantage of the valuable time needed to contain, eradicate and finally respond quickly and adequately to the incident.
Understanding the types of analysis
After this brief introduction, the following is an overview of the 4 main types of malware analysis mentioned above, addressed in a little more detail about each.
Automated Analysis with Online Sandboxes
Online services such as Any.Run, Hybrid Analysis, Virus Total, Joe Sandbox, among others, offer sandboxes that allow you to quickly observe malware behavior. These tools are useful for:
- Identify patterns quickly: Observe network connections, process creation and system modifications.
- Validate hypotheses: Confirm whether a malicious file is trying to communicate with a suspicious domain or IP address, for example.
This type of analysis, combined with good OSINT, looking for analysis reports and information on the artifact or similar samples, can resolve the incident quickly. Remember: the ideal scenario is to start with the simplest analysis and progress to more complex ones. However, it is not always feasible to use online sandboxes, especially in cases involving confidential investigations or threats directed against your organization. Listed below are some reasons to consider this limitation:
- First, there is the risk of data exposure: by submitting a sample for analysis in an online service, you may share confidential information with third parties, which compromises the privacy of the research.
- There is malware prepared for this: public sandboxes have a series of fingerprints that can facilitate their identification by some types of malware, which will certainly adjust their behavior to avoid detection.
- Network and environment constraints: online sandboxes do not perfectly replicate the victim’s real environment, which can lead to incomplete or inaccurate results.
🔗 Tip: Whenever possible, prioritize local sandboxes or controlled environments in your organization, especially for critical cases.
Static properties analysis of the binary
Coming out of a fully automated analysis scenario where third party online tools are almost always used, static property analysis can be the first scan performed in the analyst’s environment. It can be considered a safe initial examination, as it allows the analyst to explore the file without executing it, ensuring initial discoveries with low risk to the environment. Some objectives of this stage include obtaining clues in text strings (URLs, IP addresses, suspicious commands and functions in the binary) and exploring its internal structure: library imports, unconventional sections and embedded and/or compressed resources. This approach allows to quickly identify useful information about the artifact and make some initial deductions, such as:
- Identify the purpose of the file:
- Internal strings can reveal malicious functions, command and control (C2) server URLs, or even keywords indicating specific activities.
- Find out if the file really is a malicious artifact:
- Checking the file hash on services such as VirusTotal can indicate whether it is malware or an already known and widely spread variant.
- Identify binary dependencies:
- Inspecting libraries, APIs and imported functions helps to identify the expected behavior of the file and the functions it can execute.
In summary, static property analysis can help speed up the incident response process in the following ways:
- Identification of Indicators of Commitment (IoCs):
- Obtain file hashes (MD5, SHA256) and compare them with known malware
- Identification of strings with network addresses, malicious domains or commands used by malware, as well as DLL imports and API calls:
- Imported functions may suggest behaviors such as lateral movement or data exfiltration.
- Guidance for next steps:
- It is possible to identify whether the malware has features such as compression or obfuscation, to decide whether further analysis is required.
- Safety of the environment:
- By preventing the execution of malware, the risk of compromising the research environment is minimized. This is essential, especially in cases where malware can be highly destructive or exploit vulnerabilities.
- Rapid documentation and collaboration:
- The results of the static analysis, such as identified chains or dependencies, can be shared in an agile manner with other research sectors to accelerate the response process, and even with other organizations in the sector, in cases that do not require confidentiality.
Tools that Facilitate the Analysis of Static Properties
There are a number of tools available that make this process more efficient. Below is a list that is far from definitive:
- For text string extraction:
- Strings (Sysinternals and Linux Distributions), Bintext.
- Structural analysis:
- PEStudio, CFF Explorer, peframe (Linux), pestr (Linux), Binwalk.
- Signatures and hashes:
- Tools for calculating hashes (md5sum, sha1sum, etc.) associated with OSINT (VirusTotal, Hybrid Analysis, etc.).
- Metadata analysis:
- ExifTool, FOCA.
Behavioral Analysis: Discovering what malware does
Unlike static property analysis, which examines the file without executing it, behavioral analysis looks at how the malware behaves when executed, providing detailed information about its actions and targets. This process, of course, must be performed in a controlled environment, with analysis labs commonly being virtualized and with their network connections isolated from the Internet and the organization’s local network.
The following is an example of how the analysis environment can be configured, using Windows and Linux virtual machines (REMnux):
In short, behavioral analysis is the process of detonating the malware sample in a controlled and monitored environment to observe its actions, interactions with the system and attempts to communicate with other devices. This approach is useful for:
- Identify persistence techniques: Such as registry key creation, scheduled services or startup scripts.
- Detect network communications: IP addresses or command and control (C2) domains that the malware attempts to contact.
- Monitor system impacts: File modifications, process creation or changes in system configurations.
- Testing hypotheses: In this type of analysis, it is possible to confirm or discard hypotheses raised in static or other types of analysis.
By simulating a realistic environment, the analyst can observe malicious behavior without putting the corporate network at risk.
In terms of its application in an incident response context, behavioral analysis is crucial, especially in situations where static analysis methods are not sufficient to understand the threat, such as:
Revealing hidden behaviors: Many pieces of malware use obfuscation or compression techniques that make them difficult to understand through static analysis.
- Identify dynamic IoCs: These can be decrypted during execution, such as IP addresses, domains and even file names created or downloaded.
- Sizing the potential impact: Assess the extent of an infection with the malicious file.
- Proactively defend: Create use cases and detection rules based on malicious artifact behavior in protection tools (IDS/IPS/EDR/XDR).
Malware Detonation Tips
Behavioral analysis is generally performed in sandboxes or virtualized (or non-virtualized) environments, but always isolated, that replicate a real system without putting the organizational network at risk. The process includes the following steps:
- Environment Configuration: Creation of virtual machines or configuration of sandboxes to isolate malware. Monitoring tools such as Wireshark and Process Monitor are also configured in this phase.
- Malware execution: The file is executed in the controlled environment so that its actions can be monitored.
- Monitoring and Logging: Tools are used to track process creation, system calls, registry changes, network traffic and other behaviors.
- Analysis of Collected Data: Data such as IP addresses contacted, files modified and processes created are analyzed to identify malicious behavior.
Behavioral Analysis Techniques
1. Process Monitoring
Process monitoring allows us to track the actions executed by the malware in relation to the creation and manipulation of processes on the system. This includes monitoring the creation of new processes, where we seek to verify whether the malware executes other binaries or scripts; monitoring process chaining, as some pieces of malware use legitimate processes as hosts to hide their execution (process hollowing); and monitoring process behavior: here we analyze the command line arguments used in process creation or CPU and memory usage.
Tools to help implement this technique:
- Procmon (Process Monitor): Monitors processes in real time, including the creation, deletion and manipulation of files and registry keys.
- Process Explorer: Provides detailed information about active processes, including network connections and thread management.
- Sysmon: Captures detailed events on process creation, including binary hashes.
- Process Hacker: Provides a detailed view of system activities, allowing hierarchical organization of processes and analysis of processes, threads, memory, services and network connections.
2. Registry Monitoring
Some malware samples frequently manipulate the system registry to ensure persistence or modify environment settings. Monitoring for these changes can reveal persistence, environment modifications and even stealth techniques through alterations to keys related to system behavior or monitoring tools.
Recommended tools:
- Procmon, mentioned above;
- Regshot: Captures “snapshots” of the registry before and after the execution of the malware for comparison.
- Autoruns: Monitors all keys related to application startup on the system.
3. Network Monitoring
Essential to detect external communication attempts made by the malware. In this step we are looking for DNS connections to try to detect domain resolution pointing to command and control (C2) servers; web connections (HTTP/HTTPS) in the form of requests to external servers, such as data exfiltration; and other protocols such as FTP, SMB or ICMP that can be used by the malware for communication or lateral movement.
Recommended tools:
- Wireshark: for capturing and analyzing network packets, allowing the identification of malicious domains, data exfiltration and other patterns;
- Fiddler: Specialized in HTTP/HTTPS traffic analysis.
- tcpdump: A lightweight alternative for capturing network traffic in real time, ideal for command-line environments.
- Zeek (formerly Bro): An IDS that provides detailed analysis of network traffic.
- Netcat: allows you to establish TCP or UDP connections, send and receive data, and perform various tasks related to communication between devices on a network.
- InetSim: simulates various services and can help provide the malicious artifact with responses for various types of requests, such as DNS, HTTP, FTP and others.
- FakeDNS: similar to INetSim, it simulates a fake DNS to respond to malware
- File Creation and Manipulation Monitoring
Malware can create, modify or delete files during execution. Monitoring this activity helps to identify persistence files, such as additional scripts or executables; data exfiltration in the form of compressed files containing stolen information; data destruction: attempts to overwrite or delete critical files.
Recommended tools:
- Procmon;
- Sysmon;
- OSQuery: Allows you to create custom queries to monitor files and related activities on the system.
Code analysis
As already mentioned, it is possible to perform excellent analysis, obtain great results and even resolve incidents with all the types of analysis presented so far. However, as also mentioned, more specific cases of more advanced malware require more advanced analysis techniques, and this is when we enter code analysis. Although it requires more skill, it is a critical stage in dealing with sophisticated malware and ensuring a thorough investigation.
This process, performed with tools such as debuggers and disassemblers, is essential to uncover hidden indicators, as more sophisticated malware may contain encrypted functionality or functionality enabled only under specific conditions, which are only identifiable with reverse engineering; more advanced malware also employs anti-analysis techniques and is able to detect when it is being monitored and alter its behavior in sandbox environments; in addition, code analysis can help to obtain advanced IoCs, such as domain generation algorithms (DGA) or obfuscation techniques.
Conclusion
In closing this post, the final message is: malware analysis is a crucial part of the incident response process, enabling security teams to quickly understand and mitigate threats. As we have discussed, combining static property analysis and behavioral analysis techniques can already be sufficient, in most cases, to fill critical gaps in the investigation.
Although more advanced analysis, such as reverse engineering, is necessary in complex scenarios, the integrated use of static and behavioral analysis already enables effective response in most incidents. These approaches help prioritize actions, create proactive defenses and understand the scope of the threat, even for analysts who have no programming or code debugging experience.
With the support of the appropriate tools mentioned above, analysts can contribute to security teams to efficiently identify, contain and mitigate threats. By investing in these methods and tools, you strengthen your security posture and maximize your ability to protect the organization.
References
ZELTSER, Lenny. How You Can Start Learning Malware Analysis. SANS Institute, 2021. Available at: https://www.sans.org/blog/how-you-can-start-learning-malware-analysis/;
SANS Institute. Malware Analysis and Reverse-Engineering Cheat Sheet. 2021. Available at: https://www.sans.org/posters/malware-analysis-and-reverse-engineering-cheat-sheet/
