Github in the wild

Github in the wild

Github is important

Oftentimes sensitive secrets stored in a target’s GitHub environment are overlooked and thus not reported in the tool output due to the limitations of automated scanning (regex, entropy searches, etc.). On the flip slide, too much information can be outputted by automated tools, making it difficult to discern true secrets from a sea of false positives.

According to the State of Secrets Sprawl on GitHub report, 85% of the leaks occur on developers’ personal repositories and only the remaining 15% within repositories owned by organizations.

Most common types of secrets exposed on GitHub

Most common types of files found exposed on GitHub

Keys for many popular APIs have distinct structures whose compromise would result in security risk

Most detected secrets were found in cryptographic or source code files

Most of the regular expressions had high precision with minimal rejection by the filter

Leakage Mindmap

[mindmap]

Some example of offensive used github

Social Media

Platform/API: Twitter

Key Type: Access Token

Regex: [1-9][0-9]+-[0-9a-zA-Z]{40}

Platform/API: Facebook

Key Type: Access Token

Regex: EAACEdEose0cBA[0-9A-Za-z]+

Platform/API: Google YouTube

Key Type: API Key

Regex: AIza[0-9A-Za-z\-_]{35}

Platform/API: Google YouTube

Key Type: OAuth ID

Regex: [0-9]+-[0-9A-Za-z_]{32}\.apps\.googleusercontent\.com

Platform/API: Picatic

Key Type: OAuth ID

Regex: sk_live_[0-9a-z]{32}

Finance

Platform/API: Stripe

Key Type: Standard API Key

Regex: sk_live_[0-9a-zA-Z]{24}

Platform/API: Stripe

Key Type: Restricted API Key

Regex: rk_live_[0-9a-zA-Z]{24}

Platform/API: Square

Key Type: Access Token

Regex: sq0atp-[0-9A-Za-z\-_]{22}

Platform/API:Square

Key Type: OAuth Secret

Regex: sq0csp-[0-9A-Za-z\-_]{43}

Platform/API: PayPal Braintree

Key Type:Access Token

Regex: access_token\$production\$[0-9a-z]{16}\$[0-9a-f]{32}

Platform/API: Amazon MWS

Key Type: Auth Token

Regex: amzn\.mws\.[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}

Communications

Platform/API: Twilio

Key Type: API Key

Regex: SK[0-9a-fA-F]{32}

Platform/API: MailGun

Key Type: API Key

Regex: key-[0-9a-zA-Z]{32}

Platform/API: MailChimp

Key Type: API Key

Regex: [0-9a-f]{32}-us[0-9]{1,2}

Platform/API: Amazon AWS

Key Type: Access Key ID

Regex: AKIA[0-9A-Z]{16}

Private Key

Key Type: RSA Private Key

Regex: —–BEGIN RSA PRIVATE KEY—– [\r\n]+(?:\w+:.+)*[\s]* (?:[0-9a-zA-Z+\/=]{64,76}[\r\n]+)+ [0-9a-zA-Z+\/=]+[\r\n]+ —–END RSA PRIVATE KEY—-

Key Type: EC Private Key

Regex: —–BEGIN EC PRIVATE KEY—– [\r\n]+(?:\w+:.+)*[\s]* (?:[0-9a-zA-Z+\/=]{64,76}[\r\n]+)+ [0-9a-zA-Z+\/=]+[\r\n]+ —–END EC PRIVATE KEY—–

Key Type: PGP Private Key

Regex: —–BEGIN PGP PRIVATE KEY BLOCK—– [\r\n]+(?:\w+:.+)*[\s]* (?:[0-9a-zA-Z+\/=]{64,76}[\r\n]+)+ [0-9a-zA-Z+\/=]+[\r\n]+= [0-9a-zA-Z+\/=]{4}[\r\n]+ —–END PGP PRIVATE KEY BLOCK—–

Key Type: General Private Key

Regex: —–BEGIN PRIVATE KEY—– [\r\n]+(?:\w+:.+)*[\s]* (?:[0-9a-zA-Z+\/=]{64,76}[\r\n]+)+ [0-9a-zA-Z+\/=]+[\r\n]+ —–END PRIVATE KEY—–

Key Type: Private API Key

Regex: 

/^[5KL][1-9A-HJ-NP-Za-km-z]{50,51}$/

5[HJK][1-9A-HJ-NP-Za-km-z]\{49\}\b’ 

[KL][1-9A-HJ-NP-Za-km-z]\{51\}\b

Methodology

Our secret collection methodology involves various phases to identify secrets with high confidence

Gcloud

https://cloud.google.com/blog/topics/public-datasets/github-on-bigquery-analyze-all-the-open-source-code

Gcloud search query:

SELECT * FROM `bigquery-public-data.github_repos.files`

  WHERE REGEXP_CONTAINS(path,r'[Regex]’)

    LIMIT 100

Github API

As GitHub does not allow regular expression searches, these targeted queries identify candidate files which are then scanned offline for secrets

curl –location –request GET ‘https://api.github.com/search/code?q=addClass’ \

–header ‘Authorization: Token 

No.Search Query
1access_token
2access_secret
3api_key
4client_secret
5consumer_secret
6customer_secret
7user_secret
8secret_key
9—–BEGIN RSA PRIVATE KEY—–
10—–BEGIN EC PRIVATE KEY—–
11—–BEGIN PRIVATE KEY—–
12—–BEGIN PGP PRIVATE KEY BLOCK—–
13AKIA
14EAA, EAACEd, EAACEdEose0cBA
15AIza
16.apps.googleusercontent.com
17sq0atp
18sq0csp
19key- 
20sk_live_
21rk_live_ 

Scan Your Org

GSIL

Monitor Github sensitive information leaks in near real time and send alert notifications.

https://github.com/FeeiCN/GSIL

Trufflehog

Find credentials all over the place

https://github.com/trufflesecurity/trufflehog

Git-Hound

Reconnaissance tool for GitHub code search. Finds exposed API keys using pattern matching, commit history searching, and a unique result scoring system.

https://github.com/tillson/git-hound

Resources

Related Content

CONNECT WITH US

FEATURED ARTICLES

Subscribe

Sign up now to receive the latest notifications and updates from Hadess.

Sign up for News & Communications

Do you want quick & free cyber-security analysis of your application?

Secure your entire workforce, including remote employees.

TRY IT FREE

FOR 15 DAYS