Welcome to nexb-skeleton’s documentation!
VulnerableCode is a free and open database of FOSS software package vulnerabilities and the tools to create and keep the data current.
It is made by the FOSS community to improve and secure the open source software ecosystem.
The existing solutions are commercial proprietary vulnerability databases, which in itself does not make sense because the data is about FOSS (Free and Open Source Software).
The National Vulnerability Database which is a primary centralized data source for known vulnerabilities is not particularly well suited to address FOSS security issues because:
It predates the explosion of FOSS software usage
It’s data format reflects a commercial vendor-centric point of view in part due to the usage of CPE to map vulnerabilities to existing packages.
CPEs are just not designed to map FOSS to vulnerabilities owing to their vendor-product centric semantics. This makes it really hard to answer the fundamental questions “Is package foo vulnerable” and “Is package foo vulnerable to vulnerability bar?”
VulnerableCode independently aggregates many software vulnerability data sources and supports data re-creation in a decentralized fashion. These data sources (see complete list here) include security advisories published by Linux and BSD distributions, application software package managers and package repositories, FOSS projects, GitHub and more. Thanks to this approach, the data is focused on specific ecosystems yet aggregated in a single database that enables querying a richer graph of relations between multiple incarnations of a package. Being specific increases the accuracy and validity of the data as the same version of an upstream package across different ecosystems may or may not be vulnerable to the same vulnerability.
The packages are identified using Package URL PURL as primary identifiers rather than CPEs. This makes answers to questions such as “Is package foo vulnerable to vulnerability bar?” much more accurate and easy to interpret.
The primary access to the data is through a REST API.
In addition, an emerging web interface goal is to support vulnerabilities data browsing and search and progressively to enable community curation of the data with the addition of new packages and vulnerabilities, and reviewing and updating their relationships.
We also plan to mine for vulnerabilities which didn’t receive any exposure due to various reasons like but not limited to the complicated procedure to receive CVE ID or not able to classify a bug as a security compromise.
Setting up VulnerableCode
First clone the source code:
git clone https://github.com/nexB/vulnerablecode.git cd vulnerablecode
Using Docker Compose
Please find the docker documentation in Docker Installation
git clone https://github.com/nexB/vulnerablecode.git && cd vulnerablecode make envfile docker-compose build docker-compose up
Go to http://localhost:8000/ on a web browser to access the web UI.
Without Docker Compose
Compiler toolchain and development files for Python and PostgreSQL
On Debian-based distros, these can be installed with:
sudo apt-get install python3-venv python3-dev postgresql libpq-dev build-essential
Create a user named
vulnerablecodeas password when prompted:
sudo -u postgres createuser --no-createrole --no-superuser --login \ --inherit --createdb --pwprompt vulnerablecode``
Create a databased named
createdb --encoding=utf-8 --owner=vulnerablecode --user=vulnerablecode \ --password --host=localhost --port=5432 vulnerablecode
Create a virtualenv, install dependencies, generate static files and run the database migrations:
make envfile python3 -m venv venv source venv/bin/activate pip install -r requirements.txt python manage.py collectstatic python manage.py migrate
For a production mode, an environment variable named
SECRET_KEY needs to be
set. The recommended way to generate this key is to use the code Django includes
for this purpose:
SECRET_KEY=$(python -c "from django.core.management import utils; print(utils.get_random_secret_key())")
You will also need to setup the ALLOWED_HOSTS array inside vulnerablecode/settings.py according to [django specifications](https://docs.djangoproject.com/en/3.2/ref/settings/#allowed-hosts). One example would be: .. code-block:: python
ALLOWED_HOSTS = [‘vulnerablecode.your.domain.example.com’]
You can specify several hosts by separating them with a comma (,)
cd etc/nix nix-shell -p nixFlakes --run "nix --print-build-logs flake check " # build & run tests
There are several options to use the Nix version:
# Enter an interactive environment with all dependencies set up. cd etc/nix nix develop > ../../manage.py ... # invoke the local checkout > vulnerablecode-manage.py ... # invoke manage.py as installed in the nix store # Test the import prodecure using the Nix version. etc/nix/test-import-using-nix.sh --all # import everything # Test the import using the local checkout. INSTALL_DIR=. etc/nix/test-import-using-nix.sh ruby # import ruby only
Keeping the Nix setup in sync
The Nix installation uses mach-nix to
handle Python dependencies because some dependencies are currently not available
as Nix packages. All Python dependencies are automatically fetched from
./requirements.txt. If the
mach-nix-based installation fails, you might
need to update
mach-nix itself and the pypi-deps-db version in use (see
Non-Python dependencies are curated in:
Make sure to install dev dependencies by running
pip install -r requirements-dev.txt
Use these commands to run code style checks and the test suite:
black -l 100 --check . python -m pytest
Some data importers use the GitHub APIs. For this, export the
environment variable with:
If you are running behind a proxy, you will need to setup the standard
See GitHub docs for instructions on how to obtain your GitHub token.
To run all data importers use:
python manage.py import --all
To list available importers use:
python manage.py import --list
To run specific importers:
python manage.py import rust npm
REST API access
Start the webserver:
python manage.py runserver
For full documentation about API endpoints use this URL:
Continuous periodic Data import
If you want to run the import periodically, you can use a systemd timer:
$ cat ~/.config/systemd/user/vulnerablecode.service [Unit] Description=Update vulnerability database [Service] Type=oneshot ExecStart=/path/to/venv/bin/python /path/to/vulnerablecode/manage.py import --all $ cat ~/.config/systemd/user/vulnerablecode.timer [Unit] Description=Periodically update vulnerability database [Timer] OnCalendar=daily [Install] WantedBy=multi-user.target
Start this “timer” with:
systemctl --user daemon-reload systemctl --user start vulnerablecode.timer