A working version of rumal_docker can now be easily installed to test the platform. This only requires you to have Docker installed on your system. Documentation on installing Rumal Docker can be found here.
Docker Python API to Replace Subprocces¶
Subproccess popen module that was originally used for launching docker scans is currently the most viable option. Docker-py has limitations in controlling docker containers as certain options cannot be used such as –rm.
It is not officially supported by docker and lacks clear and up to date documentations. Error messages can sometimes be very vague. More information about docker-py’s limitation can be found this link.
RabbitMQ Communication System¶
The previous REST API has been replaced with RabbitMQ queue messaging system using the pika python library. This communication system has been developed to handle the issues with distributed multiple backend/frontends, and specifically these problems:
- Is it possible to have multiple consumers (backends) and a single frontend?
- Is it possible to choose what backend you want to use?
- Would it be possible to have multiple publishers (frontends) as well?
When running a new scan you can now select which backend you want to run the scan on, which will send this scan via a private queue to the backend you specified. If you have no preference on which backend to runs the scan you can select ANY option which will place it on a public queue and to be picked up by the 1st available backend.
A list of backends and the location of the public ANY queue is specified on the frontend backend.conf file.
You can find more information on the communication architecture used including diagrams and explanations in our Rumal documentation.
- No connection adapters were found for http://backend:8000”/api/v1/task/
- Myscans maximum showed scans limited to 20 results
- My Scans page: Incorrect href values in table of starred scans
- Auto-scroll Up in Panels
- fdaemon: fix urljoin() to be more robust
- Results page: Change default behavior of panels/boxes
This feature was proposed later on. Its idea was to provide an easy way to build complex search queries for locating specific scans.
It is used to search within the document returned by Thug. It uses pyparsing to build a abstract syntax tree from the grammar and build the corresponding mongo dB query. Queries are built by providing a field to query from, an operator and a value, or just a value. In the latter case all text fields will be searched. Associating them with and/or’s and parentheses will create complex queries. Currently the mongo fields that are allowed to query from are url, urls, tags, timestamp and id. Operators supported are = (contains), == (exact match), ~ (regex), >, >=, <, <=.