# security.txt: A Welcome Mat or a Red Flag for Hackers? #### Boost Your Website’s Security with `security.txt` - But Beware of the Risks! Alright folks, let’s talk about something that’s been making waves in the cybersecurity world: `security.txt`. Now, before your eyes glaze over, I promise to keep things beginner-friendly. Think of `security.txt` as a digital welcome mat for those ethical hackers – the good guys who help companies find and fix security holes before the bad guys exploit them. In the old days (and sadly, still on many sites today), if a security researcher stumbled upon a vulnerability, they’d have to jump through hoops to report it. Imagine trying to find the right contact at a massive company — talk about a headache! Enter `security.txt`, a simple text file that websites can place on their servers to provide clear contact information for reporting vulnerabilities. Sounds great, right? Well, like most things in cybersecurity, it's not that black and white. #### **A little trip down memory lane:** The idea for `security.txt` was born out of frustration. Security researchers were tired of jumping through hoops to report vulnerabilities, and website owners were often unaware of security holes until it was too late. So, a group of security experts got together and said, "There has to be a better way!" And thus, `security.txt` was born. After years of development and collaboration, `security.txt` was officially recognized as an RFC (9116) in 2021. It was a big win for the security community, and it paved the way for wider adoption of this simple yet powerful tool. #### **But here’s the catch:** While `security.txt` has gained significant traction, it's still not as widely adopted as you might think. I was curious about just how many websites were actually using it, but I couldn't find any reliable statistics (not that my test was scientific in anyway). So, being the curious cybersecurity enthusiast that I am, I decided to do a little digging myself. I found a list of the top 1,000 websites (from [CloudFlare Radar](https://radar.cloudflare.com/domains)) and wrote a Python script to check for the presence of `security.txt`. (Checking 1,000 addresses is a lot of work, let me tell you!) The results? Out of those 1,000 sites, only 20% had a `security.txt` file. That's a surprisingly low number, considering the potential benefits. If you’re interested in the technical details, you can check out the script I used on my [GitHub gist](https://gist.github.com/jtoledo3970/204338b7ffafa4ac8c9d36f343f0e28a) or at the bottom of this post. This little experiment really highlights the need for greater awareness and adoption of `security.txt`. It's a simple yet powerful tool that can make a real difference in website security. #### **Why** `**security.txt**` **is a thumbs up:** * **Think efficiency:** No more endless searching for the right email address or contact form. `security.txt` puts the information front and center, making it super easy for researchers to report vulnerabilities. * **Speed is key:** The faster a vulnerability is reported, the faster it can be fixed. This means less time for the bad guys to exploit it and cause damage. * **Proactive is the name of the game:** Having a `security.txt` file shows that a company is serious about security and encourages them to be more proactive in finding and fixing vulnerabilities. * **Building trust:** Transparency is key in today’s world. By using `security.txt`, companies can build trust with their users and show that they're committed to keeping their data safe. #### **But hold on… there’s a flip side:** * **A beacon for the bad guys?** Unfortunately, having a `security.txt` file could potentially attract malicious actors. It's like putting a sign up saying, "Hey, we're trying to be secure, but we might have weaknesses!" * **Don’t cry wolf:** There’s always the risk of false reports or spam, which can overwhelm security teams and waste valuable resources. * **Too much information?** Some worry that `security.txt` could inadvertently reveal sensitive information about a company's internal security practices. * **Tech headaches:** Implementing `security.txt` correctly requires some technical know-how, and it might not be suitable for all websites. #### **So, what’s the verdict?** Like I said, it’s a double-edged sword. `security.txt` has the potential to significantly improve website security, but it's important to weigh the pros and cons carefully. If you're considering implementing it, make sure you understand the potential risks and take steps to mitigate them. **Disclaimer:** The information and opinions expressed in this blog post are solely my own and do not reflect the views of my employer. This post is intended for educational purposes only. #### Read More: * [https://krebsonsecurity.com/2021/09/does-your-organization-have-a-security-txt-file/](https://krebsonsecurity.com/2021/09/does-your-organization-have-a-security-txt-file/) * [https://blog.cloudflare.com/security-txt/](https://blog.cloudflare.com/security-txt/) * [https://www.cisa.gov/news-events/news/securitytxt-simple-file-big-value](https://www.cisa.gov/news-events/news/securitytxt-simple-file-big-value) #### Script: import concurrent.futures import time import csv import traceback import socket import requests import threading def check\_security\_txt(url): """Checks if a website has a security.txt file.""" try: \# Enforce https:// and lowercase domain if not url.startswith(("http://", "https://")): url = "https://" + url url = url.lower() response = requests.get( f"{url}/.well-known/security.txt", timeout=10, headers={"User-Agent": "Security Scanner Bot"} ) return response.status\_code == 200 except requests.exceptions.Timeout: \# Suppress timeout error messages with open("security\_txt\_errors.log", "a") as error\_log: error\_log.write(f"{time.strftime('%Y-%m-%d %H:%M:%S')} - TimeoutError for {url}\\n") return False except requests.exceptions.ConnectionError as e: try: if isinstance(e.args\[0\].reason, socket.gaierror): \# Suppress DNS resolution error messages with open("security\_txt\_errors.log", "a") as error\_log: error\_log.write(f"{time.strftime('%Y-%m-%d %H:%M:%S')} - DNS resolution error for {url}: {e.args\[0\].reason}\\n") else: \# Suppress connection error messages with open("security\_txt\_errors.log", "a") as error\_log: error\_log.write(f"{time.strftime('%Y-%m-%d %H:%M:%S')} - Connection error for {url}: {e}\\n") except (AttributeError, IndexError): \# Suppress connection error messages with open("security\_txt\_errors.log", "a") as error\_log: error\_log.write(f"{time.strftime('%Y-%m-%d %H:%M:%S')} - Connection error for {url}: {e}\\n") return False except requests.exceptions.SSLError as e: \# Suppress SSL error messages with open("security\_txt\_errors.log", "a") as error\_log: error\_log.write(f"{time.strftime('%Y-%m-%d %H:%M:%S')} - SSL error for {url}: {e}\\n") return False except requests.exceptions.RequestException as e: \# Suppress client error messages with open("security\_txt\_errors.log", "a") as error\_log: error\_log.write(f"{time.strftime('%Y-%m-%d %H:%M:%S')} - ClientError for {url}: {e}\\n") return False except Exception as e: \# Suppress unexpected error messages with open("security\_txt\_errors.log", "a") as error\_log: error\_log.write(f"{time.strftime('%Y-%m-%d %H:%M:%S')} - Unexpected error for {url}: {e}\\n") traceback.print\_exc() return False def main(): print("Starting script...") try: top\_1m\_websites = \[\] with open("top-1m.csv", "r") as f: print("Opened top-1m.csv") next(f) \# Skip the header row for line in f: line = line.strip() try: top\_1m\_websites.append(line.split(",")\[1\]) except IndexError: top\_1m\_websites.append(line) print("Loaded websites:", len(top\_1m\_websites)) total\_websites = len(top\_1m\_websites) count = 0 positive\_results = \[\] negative\_results = \[\] results = \[\] with concurrent.futures.ThreadPoolExecutor(max\_workers=100) as executor: futures = \[executor.submit(check\_security\_txt, url) for url in top\_1m\_websites\] for i, future in enumerate(concurrent.futures.as\_completed(futures)): results.append(future.result()) if results\[-1\]: count += 1 percentage = (count / (i + 1)) \* 100 print(f"Scanned: {i+1}/{total\_websites} - Percent with security.txt: {percentage:.2f}%", end='\\r') with open("security\_txt\_log.txt", "w") as log\_file: log\_writer = csv.writer(log\_file) for website, has\_security\_txt in zip(top\_1m\_websites, results): log\_writer.writerow(\[website, "Found" if has\_security\_txt else "Not Found"\]) for website, has\_security\_txt in zip(top\_1m\_websites, results): if has\_security\_txt: positive\_results.append(website) else: negative\_results.append(website) with open("security\_txt\_positive.csv", "w", newline="") as positive\_csv, \\ open("security\_txt\_negative.csv", "w", newline="") as negative\_csv: positive\_writer = csv.writer(positive\_csv) negative\_writer = csv.writer(negative\_csv) positive\_writer.writerows(\[\[website\] for website in positive\_results\]) negative\_writer.writerows(\[\[website\] for website in negative\_results\]) percentage = (count / total\_websites) \* 100 print(f"\\nFinished checking {total\_websites} websites.") print(f"Final percentage with security.txt: {percentage:.2f}%") print("Log file saved as 'security\_txt\_log.txt'") print("Positive results saved as 'security\_txt\_positive.csv'") print("Negative results saved as 'security\_txt\_negative.csv'") except FileNotFoundError: print("Error: top-1m.csv not found.") except Exception as e: print(f"An unexpected error occurred: {e}") traceback.print\_exc() if \_\_name\_\_ == "\_\_main\_\_": try: main() except Exception as e: print(f"An unexpected error occurred: {e}") traceback.print\_exc()