API Documentation

/scrape Endpoint

The /scrape endpoint allows you to scrape a webpage and convert it into various formats. Below are the details on how to use this endpoint.

Endpoint

GET /scrape/{format}/{api_key}/{url}

Parameters

  • format: The format in which you want the scraped content. Options are json, markdown, csv, xml, and tsv.
  • api_key: Your API key for authentication.
  • url: The URL of the webpage you want to scrape.

Response

The response will contain the scraped content in the requested format.

Examples

fetch('https://ragminer.dev/scrape/json/YOUR_API_KEY/https://example.com', {
    method: 'GET',
    headers: {
        'Content-Type': 'application/json'
    }
})
.then(response => response.json())
.then(data => console.log(data));


import requests

url = 'https://ragminer.dev/scrape/json/YOUR_API_KEY/https://example.com'
headers = {
    'Content-Type': 'application/json'
}

response = requests.get(url, headers=headers)
print(response.json())


import axios from 'axios';

const url = 'https://ragminer.dev/scrape/json/YOUR_API_KEY/https://example.com';
const headers = {
    'Content-Type': 'application/json'
};

axios.get(url, { headers })
    .then(response => console.log(response.data))
    .catch(error => console.error(error));


using System;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Threading.Tasks;

class Program
{
    static async Task Main(string[] args)
    {
        var url = "https://ragminer.dev/scrape/json/YOUR_API_KEY/https://example.com";
        var client = new HttpClient();

        var response = await client.GetAsync(url);
        var responseData = await response.Content.ReadAsStringAsync();
        Console.WriteLine(responseData);
    }
}


import Foundation

let url = URL(string: "https://ragminer.dev/scrape/json/YOUR_API_KEY/https://example.com")!
var request = URLRequest(url: url)
request.httpMethod = "GET"
request.setValue("application/json", forHTTPHeaderField: "Content-Type")

let task = URLSession.shared.dataTask(with: request) { data, response, error in
    if let data = data {
        let jsonResponse = try! JSONSerialization.jsonObject(with: data, options: [])
        print(jsonResponse)
    }
}

task.resume()


curl -X GET https://ragminer.dev/scrape/json/YOUR_API_KEY/https://example.com \
-H "Content-Type: application/json"

Error Handling

If an error occurs, the API will return a JSON response with a message field describing the error.

{
    "message": "Error description"
}