Github is important
Oftentimes sensitive secrets stored in a target’s GitHub environment are overlooked and thus not reported in the tool output due to the limitations of automated scanning (regex, entropy searches, etc.). On the flip slide, too much information can be outputted by automated tools, making it difficult to discern true secrets from a sea of false positives.
According to the State of Secrets Sprawl on GitHub report, 85% of the leaks occur on developers’ personal repositories and only the remaining 15% within repositories owned by organizations.
Most common types of secrets exposed on GitHub
Most common types of files found exposed on GitHub
Keys for many popular APIs have distinct structures whose compromise would result in security risk
Most detected secrets were found in cryptographic or source code files
Most of the regular expressions had high precision with minimal rejection by the filter
Leakage Mindmap
[mindmap]
Some example of offensive used github
Social Media
Platform/API: Twitter
Key Type: Access Token
Regex: [1-9][0-9]+-[0-9a-zA-Z]{40}
Platform/API: Facebook
Key Type: Access Token
Regex: EAACEdEose0cBA[0-9A-Za-z]+
Platform/API: Google YouTube
Key Type: API Key
Regex: AIza[0-9A-Za-z\-_]{35}
Platform/API: Google YouTube
Key Type: OAuth ID
Regex: [0-9]+-[0-9A-Za-z_]{32}\.apps\.googleusercontent\.com
Platform/API: Picatic
Key Type: OAuth ID
Regex: sk_live_[0-9a-z]{32}
Finance
Platform/API: Stripe
Key Type: Standard API Key
Regex: sk_live_[0-9a-zA-Z]{24}
Platform/API: Stripe
Key Type: Restricted API Key
Regex: rk_live_[0-9a-zA-Z]{24}
Platform/API: Square
Key Type: Access Token
Regex: sq0atp-[0-9A-Za-z\-_]{22}
Platform/API:Square
Key Type: OAuth Secret
Regex: sq0csp-[0-9A-Za-z\-_]{43}
Platform/API: PayPal Braintree
Key Type:Access Token
Regex: access_token\$production\$[0-9a-z]{16}\$[0-9a-f]{32}
Platform/API: Amazon MWS
Key Type: Auth Token
Regex: amzn\.mws\.[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}
Communications
Platform/API: Twilio
Key Type: API Key
Regex: SK[0-9a-fA-F]{32}
Platform/API: MailGun
Key Type: API Key
Regex: key-[0-9a-zA-Z]{32}
Platform/API: MailChimp
Key Type: API Key
Regex: [0-9a-f]{32}-us[0-9]{1,2}
Platform/API: Amazon AWS
Key Type: Access Key ID
Regex: AKIA[0-9A-Z]{16}
Private Key
Key Type: RSA Private Key
Regex: —–BEGIN RSA PRIVATE KEY—– [\r\n]+(?:\w+:.+)*[\s]* (?:[0-9a-zA-Z+\/=]{64,76}[\r\n]+)+ [0-9a-zA-Z+\/=]+[\r\n]+ —–END RSA PRIVATE KEY—-
Key Type: EC Private Key
Regex: —–BEGIN EC PRIVATE KEY—– [\r\n]+(?:\w+:.+)*[\s]* (?:[0-9a-zA-Z+\/=]{64,76}[\r\n]+)+ [0-9a-zA-Z+\/=]+[\r\n]+ —–END EC PRIVATE KEY—–
Key Type: PGP Private Key
Regex: —–BEGIN PGP PRIVATE KEY BLOCK—– [\r\n]+(?:\w+:.+)*[\s]* (?:[0-9a-zA-Z+\/=]{64,76}[\r\n]+)+ [0-9a-zA-Z+\/=]+[\r\n]+= [0-9a-zA-Z+\/=]{4}[\r\n]+ —–END PGP PRIVATE KEY BLOCK—–
Key Type: General Private Key
Regex: —–BEGIN PRIVATE KEY—– [\r\n]+(?:\w+:.+)*[\s]* (?:[0-9a-zA-Z+\/=]{64,76}[\r\n]+)+ [0-9a-zA-Z+\/=]+[\r\n]+ —–END PRIVATE KEY—–
Key Type: Private API Key
Regex:
/^[5KL][1-9A-HJ-NP-Za-km-z]{50,51}$/
5[HJK][1-9A-HJ-NP-Za-km-z]\{49\}\b’
[KL][1-9A-HJ-NP-Za-km-z]\{51\}\b
Methodology
Our secret collection methodology involves various phases to identify secrets with high confidence
Gcloud
Gcloud search query:
SELECT * FROM `bigquery-public-data.github_repos.files`
WHERE REGEXP_CONTAINS(path,r'[Regex]’)
LIMIT 100
Github API
As GitHub does not allow regular expression searches, these targeted queries identify candidate files which are then scanned offline for secrets
curl –location –request GET ‘https://api.github.com/search/code?q=addClass’ \
–header ‘Authorization: Token
| No. | Search Query |
| 1 | access_token |
| 2 | access_secret |
| 3 | api_key |
| 4 | client_secret |
| 5 | consumer_secret |
| 6 | customer_secret |
| 7 | user_secret |
| 8 | secret_key |
| 9 | —–BEGIN RSA PRIVATE KEY—– |
| 10 | —–BEGIN EC PRIVATE KEY—– |
| 11 | —–BEGIN PRIVATE KEY—– |
| 12 | —–BEGIN PGP PRIVATE KEY BLOCK—– |
| 13 | AKIA |
| 14 | EAA, EAACEd, EAACEdEose0cBA |
| 15 | AIza |
| 16 | .apps.googleusercontent.com |
| 17 | sq0atp |
| 18 | sq0csp |
| 19 | key- |
| 20 | sk_live_ |
| 21 | rk_live_ |
Scan Your Org
GSIL
Monitor Github sensitive information leaks in near real time and send alert notifications.
Trufflehog
Find credentials all over the place
Git-Hound
Reconnaissance tool for GitHub code search. Finds exposed API keys using pattern matching, commit history searching, and a unique result scoring system.
Resources
- https://www.csoonline.com/
- https://www.ndss-symposium.org/wp-content/uploads/2019/02/ndss2019_04B-3_Meli_paper.pdf
- https://book.hacktricks.xyz/generic-methodologies-and-resources/external-recon-methodology/github-leaked-secrets
