Automated Discovery of Hidden Website API Endpoints Using Regex and AI

Overview

This workflow automates the process of discovering hidden API endpoints within websites by leveraging regex pattern matching and AI-powered output parsing. It systematically fetches website HTML, extracts JavaScript file URLs, and analyzes their content to identify potential API endpoints.

Key Features

• Automated HTML and JS Fetching: Initiates with a manual trigger and configurable settings to fetch website HTML and associated JavaScript files.
• Regex & AI Analysis: Utilizes regex and an AI output parser to extract and validate API endpoints from JavaScript content.
• Filtering & Relevance: Filters out irrelevant files and endpoints, ensuring only actionable data is retained.
• Scalable Processing: Handles multiple files and endpoints efficiently with split and filter nodes.

Benefits

• Time Savings: Eliminates manual inspection of website code, accelerating security audits and integration research.
• Accuracy: AI-enhanced parsing reduces false positives and uncovers endpoints missed by basic regex.
• Actionable Insights: Provides a clear list of hidden endpoints for further analysis or integration.

Use Cases

• Security teams conducting penetration testing or vulnerability assessments.
• Developers seeking undocumented APIs for integration.
• Analysts mapping third-party service interactions.

Integrations & Automation

Key integrations include HTTP requests, HTML parsing, and AI output parsing via LangChain nodes, all orchestrated within n8n for seamless automation.