🛠️ OPSE Part 1: A search engine for people

Introduction

In this article, we’ll present OPSE, a tool that we developed during our school project to automate the OSINT process.

This article is part of a three-part series on the OPSE project. In this series of articles we will present different part of the project:

1. Presentation of the tool w/ installation, usage and an example;
2. A technical presentation of the tool, how it is thinked and coded;
3. A guide to develop your own OPSE plugin !

Context

During our fourth year in engeneering school, we led a project on the OSINT theme. We asked ourselves how we could obtain as much personnal informations as possible on someone with few data at the beginning.

We started to imagine an automatic tool that, with few input data such as firstname and lastname, would be able to find many connections of this person on the internet. But we wanted it to be an help for anyone else in the future, so they can reuse it and modify it.

Our objectives were simple:

• automatic;
• fast;
• modular;
• open-source.

This project is now open-source and available on Github: https://github.com/OPSE-Developers/OPSE-Framework

You want to contribute on the OPSE project ? Join the Discord server by clicking on the logo !

⚠️ Disclaimer

OPSE project is made for educational & awareness use only !

⚙️ How does it work?

OPSE is a Python written project that aim to gather informations of a target by entering some basic intel of the targeted person. The project use multiple tools based on the provided input data.

Input data

Input data supported by OPSE are listed below:

• First name;
• Last name;
• Birthdate;
• Age;
• Personnal address;
• Phone number;
• Email address;
• Pseudos.

Each informations can be provided to the launching script and will start the research.

The OPSE process

With input data, according to those provided, the OPSE process calls for different tools.

Each tool takes one or more input data and return one or more data according to the results found on the targeted person. Obtaining data is different for each tool, however the method remains similar:

1. Request the resource to obtain a brute result;
2. Analyse the answer and parse interesting data;
3. Format data to be insert into profiles;
4. Highlight the most important data.

Once all tools have been called, generated profiles will be automatically fused thanks to possible links.

For example, we get as a result two profiles. The first profile return an email address and pseudos. The second profile return the same email address, a first name and a last name. As an email address is unique, we can then say that it is probably the same person. So we fuse both profiles to a new one, with fused informations.

Schema describing the OPSE fuse process

After data validation, all tools are relaunched with the new generated profiles and newly obtained data. The cycle continues until the tool have used all possible data.

Then, all generated profiles are presented to the user.

Additionnal informations

The project is working on both Linux and Windows system.

In OPSE we call plugins tool that did the research. It is possible for anyone to develop a plugin to be added on the project core. If the developer follows our rules, his plugin will be automatically loaded into OPSE. This process will be explained on the last article of this series.

For now, basic tools are implemented, the project is open for contribution. We did an hard work on the OPSE core, to make it as modular as possible. We have not yet developed a large number of plugins.

🔧 Prerequisites installation

In order to use OPSE, two package are required:

• Python & Python-pip
• Git

OPSE is developed in python so Python and his packet manager pip are required.

Git is necessary to get all repository of the OPSE project (Plugins & Core).

Linux - Prerequisites installation

Python installation

Python installation commands for Linux:

1
2
3

sudo apt-get update
sudo apt-get install python3 -y
sudo apt-get install python3-pip -y

⚠️ Python version >3.8 is required !

Git installation

Git package installation for Linux:

1
2

sudo apt-get update
sudo apt-get install git -y

Windows - Prerequisites installation

Python installation

For Windows, the Python installation executable can be downloaded from the Python website at the following address:

• https://www.python.org/downloads/

⚠️ During installation, don’t forget to check the Add Python 3.X to PATH box. It will add python executable in your variable PATH.

Git installation

Git executable can be downloaded at the following address:

• https://git.scm.com/downloads

🔧 OPSE Installation

Get OPSE Repository

First, you will need to get OPSE repositories. It is possible with the git command:

1
2

mkdir opse; cd opse
git clone --recurse-submodules https://github.com/OPSE-Developers/OPSE-Framework.git

The --recurse-submodules option allows to clone all plugins in the /tools/ directory.

Once you got the repositories, you have to install OPSE requirements.

1

python3 -m pip install -r requirements.txt

:white_check_mark: Requirements are now checked at the start of the tool. It will check that all requirements (OPSE core & plugins) are installed on the host.

Once requirements are installed you can launch ./opse.py launcher.

🚀 Launch OPSE

CLI Mode

OPSE CLI mode can be launched with the following commands:

1
2
3

cd OPSE-Framework
./opse.py <options>
./opse.py --help # show all options

GUI Mode

In GUI mode, the launcher starts the OPSE API and browse the user to the OPSE registration page. API waits for the user to register.

1
2

# Launch OPSE in GUI mode
./opse.py --api

This will browse you at the OPSE registration page.

OPSE usage

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

$ ./opse.py --help
usage: opse.py [-h] [-D] [-V] [-S] [-A]
               [-f FIRSTNAME] [-m MIDDLENAME [MIDDLENAME ...]]
               [-l LASTNAME] [-g {female,male}] [-b BIRTHDATE]
               [-a AGE] [-d ADDRESS] [-e EMAIL [EMAIL ...]]
               [-p PHONE [PHONE ...]] [-u USERNAME [USERNAME ...]]

Simple commands to deploy OPSE containers.

options:
  -h, --help            show this help message and exit
  -D, --debug           Enable debug mode
  -V, --version         Print script version and exit
  -S, --strict          Disable strict mode
  -A, --api             Launch the API only
  
  -f FIRSTNAME, --firstname FIRSTNAME
                        Specify target's firstname
  -m MIDDLENAME [MIDDLENAME ...], --middlename MIDDLENAME [MIDDLENAME ...]
                        Specify target's middlename
  -l LASTNAME, --lastname LASTNAME
                        Specify target's lastname
  -g {female,male}, --gender {female,male}
                        Specify target's gender.
  -b BIRTHDATE, --birthdate BIRTHDATE
                        Specify target's date of birth. Format: <YYYYMMDD>
  -a AGE, --age AGE     Specify target's age
  -d ADDRESS, --address ADDRESS
                        Specify target's address
  -e EMAIL [EMAIL ...], --email EMAIL [EMAIL ...]
                        Specify target's email address
  -p PHONE [PHONE ...], --phone PHONE [PHONE ...]
                        Specify target's phone number. Format: <+33XXXXXXXXX>
  -u USERNAME [USERNAME ...], --username USERNAME [USERNAME ...]
                        Specify target's username

Implementation:
  Version      opse.py X.X.X
  Authors      OPSE Developpers
  Copyright    Copyright (c) OPSE 2021-2023
  License      OPSE License

👾 Let’s try it!

Example: CLI Mode

For this example, we take as input the firstname, lastname & email address of the target:

• Firstname: John
• Lastname: Doe
• Email: jdoe@gmail.com

Launching the research

First, we see a list of all plugins loaded for the research. Loaded plugins are those that take as input the information of the target.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

$ ./opse.py --firstname john --lastname doe --email jdoe@gmail.com

[OK] Starting new research...
[OK] Plugin DeathTool has been loaded.
[OK] Plugin FacebookTool has been loaded.
[OK] Plugin HoleheTool has been loaded.
[OK] Plugin InstantUsernameTool has been loaded.
[OK] Plugin RecordsTool has been loaded.
[OK] Plugin TwitterTool has been loaded.
[OK] Successfull research: 1 profile found.

The OPSE Menu

Then, we the research is done, we face the OPSE menu with multiple entries:

1. Display profiles This menu shows all profiles that have been created during the process.

2. Merge datas This menu is used to fuse multiple profiles. For example, you found 2 website account and you know that it’s the same person, you can fuse both profile.

3. Remove profiles This option allows you to delete some profiles.

4. Research enrichment This can re-run a research based on informations that we found on the first research. For example, we entered as input the name of a target and the first research return his email address, the second research will take that email address as input.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

Menu

[0] Display profiles
[1] Merge datas
[2] Remove profiles
[3] Research enrichment

[4] Quit

Choice : 0

Result of a research

Here we take the first option to show generated profiles. Here, the research as created 1 profile with 16 websites account.

1
2
3
4
5
6

Display profile : 
 - [0] Profile : john DOE, 1 emails, 16 Website

 - [1] Return

Choice : 0

We can take the profile to show more details. As explained before, by default, as research may show sensible content on the target, we hide the result. We just know that the research returned 16 websites to which the email address is linked.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

[*]  Profile : john DOE, 1 emails, 16 Website 0x7f0487763580
 |--- Id: ba3738bbe1c94d31ae36b753656d89f5
 |--- [*] Firstname 0x7f0480185980
 |     |--- Type: Str
 |     |--- Data source: USER INPUT
 |     \--- Str value: john
 |    
 |--- [*] Lastname 0x7f04802fe940
 |     |--- Type: Str
 |     |--- Data source: USER INPUT
 |     \--- Str value: doe
 |    
 |--- Middlenames: 0 middlename found.
 |--- Usernames: 0 username found.
 |--- Phone numbers: 0 phone number found.
 |--- [+] Emails: (1)
 |     \--- [*] Email 0x7f048792bcc0
 |           |--- Type: Str
 |           |--- Data source: USER INPUT
 |           \--- Str value: jdoe@gmail.com
 |          
 |--- Accounts: 16 accounts found.  <================================
 |--- Pictures: 0 picture found.
 |--- Ips: 0 ip found.
 |--- Addresses: 0 addresse found.
 |--- Locations: 0 location found.
 \--- Organizations: 0 organization found.


Press any key to return.

If we want to show a more detailed profile, we can set Account to visible in the Update data visibility menu. Then, go back into the profile.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

[*]  Profile : john DOE, 1 emails, 16 Website 0x7f0487763580
 |--- Id: ba3738bbe1c94d31ae36b753656d89f5
 [...]
 |--- [+] Emails: (1)
 |     \--- [*] Email 0x7f0480185980
 |           |--- Type: Str
 |           |--- Data source: USER INPUT
 |           \--- Str value: jdoe@gmail.com
 |          
 |--- [+] Accounts: (16)
 |     |--- [*] Account 0x7f0487760540
 |     |     |--- Type: WebsiteAccount
 |     |     |--- Url: twitter.com
 |     |     |--- [*] Login 0x7f0487d30e80
 |     |     |     |--- Type: Str
 |     |     |     |--- Data source: USER INPUT
 |     |     |     \--- Str value: jdoe@gmail.com
 |     |     |    
 |     |     \--- Website name: twitter
 [...]

We can now see all websites linked to that email address.

Example: GUI Mode

For this example, I take as input firstname, lastname and 2 usernames.

⚠️ In this example, data has been blurred

Launching OPSE in GUI mode

To launch the OPSE API you can use the --api option:

1

./opse.py --api

It will take you directly to the OPSE registration page. The start button allows OPSE to generate a unique search ID and pass it to the API.

OPSE Registration page

Launching the research

If the registration was successful you now face the OPSE search page. Here, you can enter all intels you already have on the target.

OPSE search page

Results of the research

Now, you face a page with all profiles that have been returned by the research.

OPSE choice page

Here you can see all profiles and a small summary of each profile. You can enter a profile by clicking on the profile you want. It is also possible to select multiple profiles and then choose to manually merge or delete them.

By clicking on a profile, you will see all the profile information that has not been hidden.

OPSE results page - profile information

There is also two button:

• Export: allows to export all profile intels (csv or json)
• Delete: allows to delete the profile from the research

⚠️ The export function will be available soon

Then, if the profile return addresses, those will be mark on the following maps menu.

OPSE results page - maps

And we are done ! :)

This is a simple example of how the OPSE cli and gui work.

📖 In the next article…

In this article, we explain you how OSINT People Search Engine work in surface. We presented to you how to use it with a simple example.

In the next article, we’ll go deeper in the OPSE core. We’ll present the technical development of the tool. It will explain how core is builded, how plugins can be automatically loaded and how OPSE interface work.

see: 🛠️ OPSE Part 2: Developing an OSINT framework in Python