Ingesting FireEye Events to Threat Intelligence Systems in OpenTPX

Posted by LookingGlass TPX Team on February 24th, 2016

As we have described, OpenTPX is a JSON-based schema to allow the definition of all relevant threat context to share across systems. To aid in showing the flexibility of OpenTPX we examined FireEye event messaging and how we could ingest those events into Threat Intelligence systems.

FireEye supports different events from their products:

  • Domain Match
  • Infection Match
  • Malware Callback
  • Malware Object
  • Web Infection

FireEye events contain data relevant to the detection of a malicious website, file, or email attempting to exploit a client's browser, operating system, or application. When one of these events is detected, the FireEye appliance will generate an alert in one of several formats including JSON, CSV, XML, CEF, or LEEF.

These alerts contain a wide variety of data ranging from files or registry keys that are created, changed, or removed to network host and callback information.

We will share two examples on how those events are mapped to OpenTPX.1

Example #1: Domain Match Event Mapping to OpenTPX

Domain Match events identifies the detection of malware on a domain. The original FireEye syntax is shown here:

{
  "msg": "concise",
  "product": "Web MPS",
  "version": "7.4.1.268147",
  "appliance": "trwmps",
  "alert": {
	"src": {
  	"ip": "75.167.189.178",
  	"host": "75-167-189-178.bois.qwest.net",
  	"vlan": "0"
	},
	"severity": "minr",
	"alert-url": "https://10.2.232.117/event_stream/events_for_bot?ev_id=401&lms_iden=00:25:90:4D:B0:1A",
    "explanation": {
      "malware-detected": {
        "malware": {
          "name": "InfoStealer.Banker.Zbot.DNS"
    	}
  	}
    },
	"occurred": "2014-10-12 18:20:44+00",
	"action": "notified",
	"id": "401",
	"name": "domain-match"
  }
}

Step #1: OpenTPX Source

The first aspect of OpenTPX mapping of FireEye events is to identify the source of information. In this case, we would identify FireEye as the source of this information and when this information source is being provided.

It's possible that multiple events from a FireEye appliance could report in a single package to a server all events at a specific time.

  "source_description_s": "Alerts produced by FireEye Web MPS",
  "last_updated_t": 1446147441,
  "distribution_time_t": 1446147441,
  "schema_version_s": "2.2.0",
  "provider_s": "FireEye"

Step #2: Observable dictionary definition

The second aspect of OpenTPX mapping is to define the observable definition being used. This is typically only required for the 1st instance of an observation to an IP or Domain to define the attributes of the event. This information contains the criticality & classification information associated with the observable, and any attributes associated with this event.

"observable_dictionary_c_array": [
{
"observable_id_s": "FireEye Domain Match InfoStealer.Banker.Zbot.DNS",
  	"description_s": "An action on the network has triggered a FireEye alert.",
  	"criticality_i": 50
   	"classification_c_array": [
    	{
          "score_i": 50,
          "classification_id_s": "Actions"
    	},
    	{
          "score_i": 50,
          "classification_id_s": "Malware"
    	}
  	],
      "attribute_c_map": {
        "alert_severity_s": "minr",
        "application_name_s": "InfoStealer.Banker.Zbot.DNS",
        "src_vlan_s": "0",
        "alert_action_s": "notified",
        "alert_url_s": "https://10.2.232.117/event_stream/events_for_bot?ev_id=401&lms_iden=00:25:90:4D:B0:1A",
        "src_host_fqdn_s": "75-167-189-178.bois.qwest.net",
        "src_ip_ipv4_s": "75.167.189.178"
  	},
}

Step #3: Observable Association

The final step of OpenTPX mapping is to define the list of IPs or Domains that have been associated with this observable and when those events occurred. The example below shows 1 IP address and 1 domain associated with the Observable InfoStealer.Banker.Zbot.DNS

"element_observable_c_array": [
{
"subject_ipv4_s": "75.167.189.178"
      	"threat_observable_c_map": {
    		"FireEye Domain Match InfoStealer.Banker.Zbot.DNS": {
          		"occurred_at_t": 1413152444,
          		"last_updated_t": 1446147441
    		}
  	},
},
{
      "subject_fqdn_s": "75-167-189-178.bois.qwest.net"
      "threat_observable_c_map": {
    		"FireEye Domain Match InfoStealer.Banker.Zbot.DNS": {
          		"occurred_at_t": 1413152444,
          		"last_updated_t": 1446147441
    		}
  	},
}
],

Example #2: Malware Object Event to OpenTPX Mapping

The Malware Object event conveys information associated with the malware found.

The original FireEye event example is as follows:

{
  "msg": "concise",
  "product": "Web MPS",
  "version": "7.4.1.268147",
  "appliance": "trwmps",
  "alert": {
	"src": {
  	"ip": "242.183.213.185",
  	"vlan": "0"
    },
	"severity": "majr",
	"alert-url": "https://10.2.232.117/event_stream/events_for_bot?ma_id=6&lms_iden=00:25:90:4D:B0:1A",
    "explanation": {
      "malware-detected": {
        "malware": {
          "executed-at": "2014-10-12T18:25:55Z",
          "md5sum": "c2a00731d94851ad7e25d358a55320c8",
          "type": "exe",
          "name": "Trojan.Sinowal"
    	}
  	}
	},
	"occurred": "2014-10-12T18:25:55Z",
	"id": "6",
	"action": "notified",
	"dst": {
  	"ip": "246.85.89.190"
	},
	"name": "malware-object"
  }
}

To map to OpenTPX Steps #1 and Step #2 are very similar to Example #1.

Step #1: Source definition.

The Source definition allows OpenTPX packages to identify who is sending this Threat Intelligence.

  "source_description_s": "Alerts produced by FireEye Web MPS",
  "last_updated_t": 1446147441,
  "distribution_time_t": 1446147441,
  "schema_version_s": "2.2.0",
  "provider_s": "FireEye"

Step #2: Observable Definition

The Observable definition provides the criticality, classification and metadata attributes associated with the observable. As stated in Example #1, this information is defined once for all instances of the particular observable. For systems that are continuously exchanging primarily instances of malware or threat intelligence that is occurring on newly observed IPs or Domains then this information does not need to be re-shared.

"observable_dictionary_c_array": [
{
"observable_id_s": "FireEye Malware Object Trojan.Sinowal",
 	"criticality_i": 50
  	"description_s": "An action on the network has triggered a FireEye alert.",
  	"classification_c_array": [
    	{
          "score_i": 50,
          "classification_id_s": "Actions"
    	},
    	{
          "score_i": 50,
          "classification_id_s": "Malware"
	    }
  	],
  	"attribute_c_map": {
        "dst_ip_ipv4_s": "246.85.89.190",
        "alert_severity_s": "majr",
        "application_name_s": "Trojan.Sinowal",
        "src_vlan_s": "0",
        "alert_url_s": "https://10.2.232.117/event_stream/events_for_bot?ma_id=6&lms_iden=00:25:90:4D:B0:1A",
        "malware_type_s": "exe",
        "executed_at_t": 1413152755,
        "alert_action_s": "notified",
        "hash_md5_h": "c2a00731d94851ad7e25d358a55320c8",
        "src_ip_ipv4_s": "242.183.213.185"
  	},
}

Step #3: Observable association

The observable association can associate the IP, Domain and Application Hash identifiers to the observable definitions. The example below shows 2 IP address that have been associated with the Trojan Sinowal and one malware hash identified by the MD5. It should be noted that if multiple observables were associated in the same report for the same IP, Domain or Hash then this observable map structure would be extended to include those additional observables rather than having to send additional events.

  "element_observable_c_array": [
	{
      "subject_ipv4_s": "242.183.213.185"
      "threat_observable_c_map": {
    	"FireEye Malware Object Trojan.Sinowal": {
          "occurred_at_t": 1413152755,
          "last_updated_t": 1446147441
    	}
  	},
	},
	{
      "subject_ipv4_s": "246.85.89.190"
  	"threat_observable_c_map": {
    	"FireEye Malware Object Trojan.Sinowal": {
          "occurred_at_t": 1413152755,
          "last_updated_t": 1446147441
    	}
  	},
	},
	{
      "subject_md5_h": "c2a00731d94851ad7e25d358a55320c8",
  	"threat_observable_c_map": {
    	"FireEye Malware Object Trojan.Sinowal": {
          "occurred_at_t": 1413152755,
          "last_updated_t": 1446147441
    	}
  	}
	}
  ]

In Summary

This blog has shown how to map FireEye events to OpenTPX for the ingestion of those events as Threat Intelligence into other systems.

One of the key aspects of mapping FireEye events to Threat Intelligence was to associate the events with Threat Intelligence classification and meta-attributes so that it's easier to correlate events to scoring systems consuming those events. This will hopefully show how threat analysis can then be improved by leveraging incoming FireEye events that have been categorized and scored.

We hope that you see how easy it is to use OpenTPX for real-time events into Threat Intelligence.


1 FireEye is a registered trademark of FireEye,Inc.