GRNET NOC develops a Firewall on Demand platform using Python, Netconf and taking advantage of BGP Flow Spec

A bit of theory

Security is a major concern of today’s networks and in particular reactive protection against Denial of Service attacks. To date, attack mitigation is carried out mainly by two security approaches. The first approach is to deploy old style access lists either close to the server, resulting in valuable resources already wasted, or to the ingress point of the target domain. In the latter case uplink bandwidth is also wasted while the administrative overhead and large response times together with the coarse nature of access lists renders the mitigation actions ineffective. An improvement is based on a technique known as Remote Triggered Black-Hole (RTBH). Using BGP as a security tool, the receiving router translates a BGP community into a discard Next-Hop. The coarse actions that are implemented by RTBH, render the victim unreachable to the entire Internet, thus terminating in itself the DDOS attack. The positive side of RTBH is that the network administrators have successfully mitigated the flood of inter-domain traffic but even in this case the uplink remains flooded with attack traffic. An enhancement to existing approaches would be the deployment of a security tool that would allow firewalling rules to be propagated to different domains independent of unicast routing, while allowing for n-tuple of matching and filtering actions.

We, in GRNET’s NOC, driven by the need to provide advanced security services develop our own in-house security provisioning platform using readily available Open-Source software. Our approach towards security provisioning is the usage of flow specification NLRIs to successfully convey filtering information amongst neighbouring domains.Our deployment is based on BGP family flow supporting hardware (RFC 5575). We have deployed a hardware box (Juniper EX4200) that is layer 3 capable and supports BGP and address family flow. This box is located in one of GRNET’s data centers and conveys flow information through external BGP sessions with GRNET border routers. A first (coarse) level of trust is applied on the BGP sessions and is based on the well established trust of unicast routing. To allow for automation we have mitigated a more strict authorisation model to the web front-end. Therefore a filter is applied on the routers that allow all GRNET customers rules to be accepted on the flow routing table.

Our web platform is developed in Python Django framework. The platform allows for creation, modification and deletion of flow rules via a wizard-like GUI. Authentication is performed via Shibboleth using an appropriate Entitlement attribute, carried only by NOC personnel. This gives the service a federated nature and eliminates any concern that the mechanism is a DDOS by itself. Authorization is based on address space assignments to GRNET’s customers. A number of GRNET’s clients are assigned a private AS number not allowing in this case the use of RIPE public whois. To address this issue GRNET has implemented a local private whois server where all the records are kept. Each client Institution/University has each own address space allocated by RIPE and also registered by GRNET Hostmaster in appropriate route-object located on the local whois server.

While existing bgp flow-spec approaches make usage of bgp daemons, our implementation is solely based on NETCONF. NETCONF was chosen for being a secure management protocol with clean XML structure and a well defined request/response schema. Furthermore, the hardware that the rules are applied supports NETCONF. On the backend of the tool, a python NETCONF proxy middleware translates user requests to BGP flow rules and vice-versa. The NETCONF middleware applies the produced configuration to the hardware box via a python SSH-NETCONF client. Configuration retrieval is also supported to allow for syncing and reconciliation. Device configuration is mapped to Python classes that can be easily distributed and reused among a variety of applications. Once a request for a new flow rule is placed, configuration is applied to the aforementioned hardware box and is then propagated via BGP to GRNET’s routers. What really boosts the deployment of the platform is the use of open source tools that map network configuration to service elements. This allows for interdomain deployment as tools can be easily adapted to each domain needs and requirements.

..and some action

Firewall on Demand runs as a Gunicorn gevent application. The GUI is served via Apache proxying of the WSGI application. As Django is used, Django templates serve the frontend along with some jQuery libs. Rule requests are stored in a database. Once a rule is requested and some checks are passed server-side, a new Celery job is spawned. The request is translated from python objects (model) to Netconf device configuration via nxpy. This configuration is passed to a Python NETCONF client (Shikar’s patched client) and then to the device (EX4200) via NETCONF. As Celery runs on one worker, requests are passed to the device in a serial manner, thus avoiding locks. Once the configuration is applied to the device, the device responds. The response is parsed by the “python objects to network configuration proxy” and via a long-polling mechanism is presented to the user.

The system architecture is presented right here.

 

Want to see it in action?

Login Screen

Logging in with your federation account brings you to your rules (per peer view):

Let’s create a new rule…

 

Right… submitting the form brings us to the home screen, with the new rule added as pending.

Let’s check the console. The console is updated via long polling whenever there is a status update. No page refresh is required. It works similarly to facebook’s chat and twitter’s new tweet notifications. The console is a modified jQuery dialod ;) .

Great! The request seems to have been applied to the device. It’s active!

Let’s check the device… ssh to our device and then “edit private”->”show routing-options flow”…

 

 

Awesome! Our rule has been applied to EX4200. It will be propagated via eBGP to its neighbouring router and BGP flow spec will do the rest!!

Editing is easy to accomplish

…and deleting is even easier

Each user can apply rules targeting only his administrative network

We have conducted tests on our production network. The average time before a DDOS ismitigated is 15 seconds counting from the moment that the user clicks “Apply” on the creation form.

A screencast is availiable:

Update (25-01-2012):

The platform has now moved to GRNET’s production network. Tests are conducted emulating network attacks (via iperf) between various points in GRNETs network. These attacks are mitigated successfully by Firewall on Demand. Here is a screenshot from an attack mitigation (rate-limit to 100Kbps).

Update (16/02/2012):

Authors: Leonidas Poulopoulos, Michalis Mamalis, Andreas Polyrakis

Posted in Coding, Netconf | Tagged , , , , , , | 2 Comments

Implementing a SOAP-Python-NETCONF Technology Proxy for GÉANT’s BoD platform, AutoBAHN

What is AutoBAHN?

As [1] states: “AutoBAHN stands for Automated Bandwidth Allocation across  Heterogenous Networks. It is a Bandwidth on Demand system dedicated to reserve  resources in heterogeneous, multi-domain environments, allowing immediate and  advance circuit reservations. The project was developed in GN2 as a pilot and is continued in GN3 which aims at providing the production Bandwidth on Demand service  for the Geant community.”

What is that … Technology Proxy?

Each AutoBAHN instance consists of 3 main components.

More info about the components can be found at [1].

What is interesting is that the technology proxy (TP) maps incoming requests for a circuit, to technology/vendor specific commands that, once applied to the devices, establish the requested circuit between the remote endpoints.

For the time, TP is implemented using Java and an expect-cli module. Since NETCONF can do beyond expect-cli with added-value functionalities especially for equipment that supports it, a new TP implementation with NETCONF could boost the system’s performance, along with its security (due to SSH v2).

As every module interoperates with the above using SOAP web services, and having in handy the Python ncclient [2], a new TP could have a Southbound interface implemented as a Python SOAP Server and a Northbound interface implemented as a Python NETCONF client.

This project is at the moment under heavy development. The Southbound interface complies to the TP wsdl. The Northbound interface is a modified ncclient [3]. Besides that, the middleware maps SOAP request parameters to python classes that produce NETCONF XML parts. They are then fed into the ncclient that applies the requests to the devices.

Stay tuned…

References:

[1]: https://forge.geant.net/forge/display/autobahn/Home

[2]: http://schmizz.net/ncclient/

[3]: https://github.com/leopoul/ncclient/tree/dev_junos

Posted in Coding, Random | Tagged , , , | Leave a comment

GRNET network graphs mobile flavor

We just rolled out a mobile edition of GRNET’s network graphs tool. Remember to access it from your mobile device at: http://mon.grnet.gr/rg

Tip: Add it as a shortcut at your home screen

GRNET graphs mobile flavor (iPad screenshot)

 

Posted in Coding | Tagged , , | Leave a comment

How to turn an old laptop to a digital photo frame

Recently I discovered my wife’s old Compaq laptop (bought in 2000) in our storage room. It was an HP Omnibook XE3 – PIII with 128MB memory and 10GB HD. Googling around I came up with the idea of turning it into a digital photo frame (DPF). From my Google findings everyone admitted (I agree with that) that the hardest part is the tearing apart of the laptop to its components.  Beware! The most sensitive part is the screen along with its cabling.

The laptop was shipped with Windows me, which were replaced by Puppy Linux for my project’s needs. As the laptop didn’t have WiFi NIC, I bought a PCMCI Netgear that works like a Swiss with Puppy.

Removing the screen and getting rid of the plastic motherboard casing took me approximately 3-4 hours. Placing the components into the wooden frame took me 2.30 hours. I begun with the removal of the keyboard then moved on to the removal of the screen and its cabling and then to accomplish my target way faster I …. broke gently(!!!!!) the plastic casing being very careful not to destroy the motherboard.

Motherboard (plastic casing removed)

My DPF would be hosted in a wooden frame. The perfect frame for my parts’ dimensions was the 30x40cm RIBBA from IKEA. I centered and glued my screen on the frame’ s white carton paper.

Screen centered and glued with duct tape on carton

As the duct tape might not be sufficient for the screen’s weight, I supported the screen with wood on its lower left corner. The wood would then be the foundation for the screen’s back cover. The wood was glued with wood glue on the frame. A piece of the wood was cut and placed on the top right corner to balance and stabilize (with screws) the screen wooden cover.

Screen wooden support

The frame’s thin wooden panel would be used as the screen cover and the motherboard base. I opened 3 holes on this piece of wood. The two of them were for the screen cables and the other one for the motherboard’s fan.

Frame back wooden panel drilled

I stuck 4 pieces of two-side tape at the bottom of the motherboard to make it stable on the wood and then screwed it on the wood. I chose holes on the motherboard that seemed secure and would not cause any hardware damage

Motherboard taped

Motherboard screws

When motherboard got in place, I connected the screen’s cabling to its place on the motherboard, and connected the power cord.

All connections are made

Then I prayed that my 2 and a half hours were not a waste of time…

And…my prayers were heard!!!!

Puppy linux Grub menu

Puppy booted flawlessly and the best way to test it was the flickr slideshow

Flickr slideshow

Having kept the laptop speakers I turned an old laptop into a DPF – internet radio.

Minor adjustments are in my future plans. The power button has to be unsoldered and placed into the side of the frame plus I have to buy a wireless mouse and keyboard for the DPF. Alternatively I could write a script that starts a slideshow on startup.

Try it yourself with an old laptop! It’s fascinating!

Update (24/08/2011): More guides and instructions on installing Linux on laptops and turning old laptops to DPFs can be found here:

http://tuxmobil.org/hp.html
http://tuxmobil.org/laptop_digital_picture_frame_with_linux.html

http://repair4laptop.org/disassembly_hewlett_packard.html
http://repair4laptop.org/notebook_picture_frame.html

Posted in DIY | Tagged , , , | 2 Comments

NETCONF – the cage : Juniper vs. Cisco (part 1)

It was just yesterday that a Cisco switch got my attention and was tested for its NETCONF capabilities and features. From my small NETCONF experience with Juniper equipment I soon realized that as NETCONF is Juniper’s “child”, Cisco does not seem to care that much about it. This is not only my personal opinion. Take for example the vast documentation (aprox. 3000 pages with the available rpc operations plus 200-300 pages with configuration details and clean examples) that Juniper offers compared to some 30 pages I found from Cisco.

Let’s move on to some quick comparison…

Continue reading

Posted in Coding | Tagged , , | 1 Comment

Mobile upload and porting of old blogger posts

Old posts from leopoul.blogspot were imported here plus this is a mobile upload

Posted in Random | Tagged | Leave a comment

A blog powered by GRNET’s ViMa service

It’s nice to sit back and enjoy this blog that sits on a VM powered by http://vima.grnet.gr (GRNET’s Ganeti web interface)

Posted in Random | Leave a comment

JQuery+rrd+Datetimepicker = In depth view of graphs

If you generate rrds for your network traffic, you may be faced with the question: “How can I get a granular view for those 10 minutes of high traffic two days ago?”
With a good back-end written in python and a front-end in Django templates, you can easily incorporate jQuery and then jQuery UI and then to reach your goal, just deploy Datetimepicker (http://trentrichardson.com/examples/timepicker/).

Add some change bindings in jQuery and voila!

Portion of code used:

    $(function(){
        $(“#id_start”).datetimepicker({
            dateFormat: ‘yy-mm-dd’,
            timeFormat: ‘hh:mm:ss’,
            showSecond: true,
            maxDate: ‘+0d’,
            changeMonth: true,
            changeYear: true
        }).datepicker(‘setDate’, new Date()).datepicker($.datepicker.regional["el"]);
        $(“#id_end”).datetimepicker({
            dateFormat: ‘yy-mm-dd’,
            timeFormat: ‘hh:mm:ss’,
            showSecond: true,
            maxDate: ‘+0d’,
            changeMonth: true,
            changeYear: true
        }).datepicker(‘setDate’, new Date()).datepicker($.datepicker.regional["el"]);
        var utc_start = Math.round((new Date()).getTime() / 1000);

        var utc_end = Math.round((new Date()).getTime() / 1000);
        $(“#id_start”).bind(‘change’, function(){
            utc_start = (($(this).datetimepicker(‘getDate’)));
            utc_start = Math.round((utc_start).getTime() / 1000);
            if (utc_end-utc_start < 0){
        $(‘#graphtime’).attr(‘src’, “{{MEDIA_URL}}images/errorgraph.png”);
    }
    else{
$(‘#graphtime’).attr(‘src’, “{{ graph.url }}” + utc_start + “,” + utc_end);
    }
        });
        $(“#id_end”).bind(‘change’, function(){
            utc_end = (($(this).datetimepicker(‘getDate’)));
            utc_end = Math.round((utc_end).getTime() / 1000);
    if (utc_end-utc_start < 0){
        $(‘#graphtime’).attr(‘src’, “{{MEDIA_URL}}images/errorgraph.png”);
    }
    else{
        $(‘#graphtime’).attr(‘src’, “{{ graph.url }}” + utc_start + “,” + utc_end);
   } 
    });

});

 

Posted in Coding | Leave a comment

Tough times getting Django over SQL Server 2000…the least that you can suffer

Trying to get Django to operate on a MS SQL Server 2000 can be real pain, especially if the database is filled by a commercial application and their content is encoded in Greek.

The least that you can get is to miss table rights from the auth_permission table, especcialy if you are trying to build your models on pre-existing SQL tables
This snippet saves the day:
from django.core.management import setup_environ
try:
    import settings
except ImportError:
    import sys
    sys.stderr.write(“Couldn’t find the settings.py module.”)
    sys.exit(1)

setup_environ(settings)

# Add any missing content types
from django.contrib.contenttypes.management import update_all_contenttypes
update_all_contenttypes()

# Add any missing permissions
from django.contrib.auth.management import create_permissions
from django.db.models import get_apps
for app in get_apps():
    create_permissions(app, None, 2)

Thanks paltman
Posted in Coding | Leave a comment

GRNET’s topology as iGoogle gadget

GRNET’s Topology as an iGoogle gadget, parameterized through gadget’s “Edit Settings. Click the button below to add it to your iGoogle pane:

Add to Google

Posted in Coding | Leave a comment