Parser Guidelines
Apply the following guidelines when creating a parser:
- Define test cases that exercise all parser logic.
- Personally identifiable information (PII), such as IP addresses, URLs, user names, and email addresses, are not allowed for use in packages.
- Use valid test data instead of PII. For more info, see Parser sample data.
- Avoid including third-party assets, such as company names and domain names, unless they are directly related to the test.
- Add comments that fully describe the parser logic.
- Use CPS-compliant fields to set individual parser fields. For more info, see CPS compliant parser fields.
- Create custom parsers based on the following assumptions:
- They receive one event at a time.
- Ensure events are not in bulk when they reach the parser.
- When logs are sent in bulk to the parser, the parser needs to be customized to split them into separate events.
- They receive events that are not wrapped in custom transport layers. For example, if logs are wrapped in a layer of JSON, the parser needs to be customized to remove the data wrapper.
- Use the parser template when creating parsers. For more info, see Parser template.
- Ensure that any explicitly supported transport methods align with these guidelines.
Parser sample data
Section titled “Parser sample data”We strongly recommend including sample data for tests in parsers and example data in your packages for demonstration purposes. Review the following info for guidelines on how to create good sample data for your packages. For more info, see Wikipedia’s Placeholder names.
Human names
Section titled “Human names”We encourage using John Doe and Jane Doe in sample data.
Company names
Section titled “Company names”Acme is a common company that you can use in sample data, you can also use Octan.
Domain names
Section titled “Domain names”This table presents a selection of example domain names to demonstrate typical formats and structures.
| Name | Purpose |
|---|---|
example.com | Completely generic domain that’s reserved for example data. For more info, see RFC 2606. |
example.net | Completely generic domain that’s reserved for example data. For more info, see RFC 2606. |
example.org | Completely generic domain that’s reserved for example data. For more info, see RFC 2606. |
*.example | Top level domain reserved for somewhat relatable data, such as acme.example.com |
mitre.org | Since Mitre is referenced in many security use-cases, it is okay to use in sample data. |
E-mail addresses
Section titled “E-mail addresses”E-mail addresses can include almost any combination of human names and domain names, as long as they align with general rules for email addresses. See this table for some common examples.
| Address | Purpose |
|---|---|
johndoe@example.com | A random, private person. |
janedoe@acme.example | An employee of a company named Acme. |
Generic addresses
Common generic local parts are also allowed. This list is not exhaustive, but it is meant to provide guidance.
| Address | Purpose |
|---|---|
noreply@example.com | Commonly used email company mailbox |
info@example.com | Commonly used company public mailbox |
mailer-demon@example.com | System mailbox |
postmaster@example.com | System mailbox |
abuse@example.com | Abuse mailbox |
webmaster@example.com | Homepage author mailbox |
IP addresses
Section titled “IP addresses”While IP addresses are public information, we recommend not to use any public routable IPs. The unknown history of IP addresses and lack of control over associated data usage pose potential risks. To mitigate these risks, we advise using non-routable IP addresses, ensuring they cannot be externally accessed.
Private networks
For local networks, we recommend using RFC 1918 networks.
| Block | Purpose |
|---|---|
10.0.0.0/8 | Private-Use Network |
172.16.0.0/12 | Private-Use Network |
192.168.0.0/16 | Private-Use Network |
169.254.0.0/16, fe80::/10 | Link-Local addresses |
224.0.0.0/4 | Multicast network. Typically used for devices to locate eachother behind the same firewall |
fc00::/7 | Unique Local Addresses |
ff00::/8 | Multicast Addresses RFC 4291 |
Public networks
For public networks, we generally only allow TEST-NET blocks, with a few exceptions. The list of exceptions is growing, so direct any requests for additional networks to humio_packages@crowdstrike.com.
Note: We generally only allow IPs of services that are specific to an IP address, such as DNS.
| Block | Purpose |
|---|---|
192.0.2.0/24 | TEST-NET-1. For more info, see RFC 5737. |
198.51.100.0/24 | TEST-NET-2. For more info, see RFC 5737. |
203.0.113.0/24 | TEST-NET-3. For more info, see RFC 5737. |
1.1.1.1, 1.0.0.1 | CloudFlare DNS |
8.8.8.8, 8.8.4.4 | Google DNS |
2001:db8::/32 | Reserved for documentation and examples |
2001:4860:4860::8888, 2001:4860:4860::8844 | Google DNS |
2606:4700:4700::1111, 2606:4700:4700::1001 | CloudFoundry DNS |