Our site uses cookies to make it work and to help us give you the best possible user experience. By using our site, you agree to our use of cookies. To find out more about cookies and how you can disable them, please read our cookies statement. 

Cookie Settings

You can manage your cookie settings by turning cookies on and off.

Click on the different cookie  headings to find out more about the types of cookies we use on this site and to change your settings. Please be aware that if you choose to turn off  cookies, certain areas of our site may not work and your browsing experience may be impacted.

For further information on how we use cookies, please see our cookies statement. 

Strictly Necessary Cookies

(Req)

These cookies are essential for the technical operation of and proper functioning of our site  and enable you to register and login, to easily move around our site, and to access secure areas. Without these cookies our site won't function properly.  

These cookies are required

Performance Cookies

Performance cookies allow us to collect aggregated and anonymous data on how our site is used, such as the number of visitors to our site, how you navigate around and the time spent on our site and also to identify any errors in functionality. These cookies also help us to improve the way our site works by ensuring that you can find what you’re looking for easily, to better understand what you are interested in and to measure the effectiveness of the content of our site. 

Marketing Cookies

These cookies allow us to advertise our products to you and allow us to pass this information on to our trusted third parties so that they can advertise our products to you on our behalf. All information these cookies collect is aggregated and therefore anonymous. No personal information is shared to third parties. Any personal information collected while using our website could be used for direct marketing from Dimension Data only.

Do the flow-bot: applying machine learning to internet-scale security analytics

Blog

The analysis of network flows for security is not new and has been adopted in both network and security industries for more than a decade. It was originally invented for high speed switching but has also been used in Distributed Denial-of-Service (DDoS) attack detection.

Recently, with the help of technological advancements in machine learning and streaming analytics, it is getting renewed attention as a countermeasure to rapidly evolving cyber-attacks by globally syndicated adversaries beyond DDoS attacks. Additionally, as internet traffics are encrypted at an accelerated rate, the meta information, such as network flow, is becoming the only available data for the analysis anyway.

New large-scale network analytics

By applying machine learning to the internet scale network analytics, it is possible to produce high quality, real-time blacklists of Command and Control (C&C) servers that are detected up to two weeks earlier than major vendors.

The internet scale network analytics provides a more complete understanding of botnet infrastructures as they are being formed in real-time. This includes the location and nature of C&C servers, bots under the control, and ultimately who is behind the infrastructure. It facilitates the quick detection of targeted attacks, flagging if a given attack is random (i.e., it flows to many organisations) or is targeted (i.e., flows to a specific organisation).

The analysis enables us to block and ultimately take down the botnet infrastructure from which attackers launch many types of attacks, such as DDoS and ransomware distributions.

Advancements in network flow analysis

So what is the key to effectively enhancing security network flows? Access to a large amount of flow data helps, however, advancements made in these two areas prove to be crucial:

Big streaming analytics platform

Recently, open source projects in streaming analytics, such as Apache Kafka, Spark and Flink, are becoming very active and producing innovative software. NTT Group, including companies such as NTT Security, Dimension Data and NTT DATA, is actively participating in those projects and has built a scalable, fast-streaming platform by leveraging the best of breeds of open source software. The platform can easily handle pipeline processing of hundreds of thousands of flows per second and enables us to apply advanced analytics to a large amount of data streams in a massively scalable manner.

Data quality improvement

Arguably, for machine learning, the quality of data used in training is the most important factor that determines the overall performance. Working on correlation with results from passive Domain Name System (DNS) data and flow analysis to improve accuracy and coverage is imperative to achieving that security utopia.

There also now exists the possibility of integrating with internet scale active scanning (in the IPv6 space), as well as catering for Operation Technology (OT). In OT environments, non-IP proprietary protocols are often used but flow patterns are statistically predictable. Because of these characteristics, OT environments lend themselves well to flow-based anomaly detection.

The number of ways in which IoT devices can help people and organisations is boundless. However, IoT devices pose new and unique security challenges due to their massive and ubiquitous installed base, as well as the limitation in their computing resources. Continuing to invest in enhancing large-scale network analytics is essential, not only for IoT but also other disruptive technologies.

For more information on our new botnet infrastructure detection capabilities, click here.

 

Previous Article: Breaking down the walls between applications Next Article: Innovation through collaboration: what it takes to make a workplace a ‘great place’

You may be interested in

Cityscape
Blog

Securing the multi-cloud

As economic and operational benefits of the cloud became clearer, business units aggressively drove cloud services adoption across the business.

Read blog
Computers
Blog

Now’s the time to engage a Managed Security Services provider

There are now more reasons than ever to engage a managed security services provider (MSSP).

Read blog
Blog

Automation and orchestration: transforming how cybersecurity teams work

It’s an exciting time to be in cybersecurity, as automation and orchestration help to transform how Cybersecurity teams operate on a day-to-day basis.

Read blog
Cityscape
Blog

The two-way conversation you need to have with your CEO on cybersecurity

With cyberattacks featuring regularly in mainstream news, it’s good to see that Boards and CEOs are becoming more cybersecurity conscious.

Read blog