Spotted a one-legged pirate Myna. He’s currently en route to a specialized woodpecker for a prosthetic leg upgrade after that skirmish with the one the left went south. Recovery time: three seeds and a short flight.
There are people in this world who spend their weekends hiking, cooking meals, or having conversation with their friends. And there are kind of people who looks at a number on a website and thinks: what if that number… could be somewhere else, automatically, forever? And then loses a good day of their life to it.
The “problem” was this: I wanted to display current total banking sector lending, maybe exchange rates, or any updated element of any website on my blogs inline in my posts. The normal solution is to copy the number, paste it, and update it whenever they remember. Takes thirty seconds.
But why do that? Instead lets build a full data scraping engine in PHP – complete with XPath selectors, retry logic with exponential backoff, a five-item-type calculation pipeline, circular dependency detection, a live testing interface, a stylish dashboard with sparkline charts, and a wordpress shortcode system where you could type [npr_in_billion] for total Total Deposits (in NPR Billion) and feel like an absolute wizard and an idiot at the same time.
The truly unhinged part is – being completely self aware of mundane things that we do and still keep doing it. It a juggle between “this is way too much” and then also thinking: but what if we also add 7-day historical snapshots with trend arrows? There is a specific kind of brain rot where the complexity itself becomes the gratification. The original task becomes a thin excuse to keep building.
Here is what I actually ended up with: a dashboard at a URL only I know about, showing me live data I could find in ten seconds on Google, formatted beautifully in a font, auto-refreshing every 24 hours whether I check it or not. The shortcodes work perfectly. The sparklines are delightful. I have used it in production exactly once, in a blog post, to prove it worked, and that blog post is this one.
There is a very particular satisfaction in building something clean and complete, something that works exactly the way you designed it, even if the world did not ask for it and will not notice it exists. Most real problems are messy, unfinishable and solutions unrequired.
But a self-contained tool that does its job quietly in the background? That’s the closest thing to peace in this world.
Anyway, the current total Total Deposits (in NPR Billion) USD to NPR rate is [usd_npr] whose shortcode is [ usd_npr ]. That updated itself automatically. You’re welcome.
Okay, here is the tool working practically.
[npr_billion] – used [ npr_billion ] shortcode
[data name=”npr_billion”] – used [ data name=”npr_billion” ]
engine.php
<?php
/**
* ╔═══════════════════════════════════════════════════════════╗
* ║ DataEngine v1.1 — engine.php ║
* ║ Dynamic Data Scraping, Calculation & Dashboard System ║
* ║ Runs standalone inside any PHP / WordPress environment ║
* ║ ║
* ║ v1.1: Added regex & json_path extraction methods ║
* ║ for JS-rendered sites (e.g. ocr.gov.np) ║
* ╚═══════════════════════════════════════════════════════════╝
*
* Place at: /api/engine.php
* Storage: /api/data.json (auto-created)
*/
// ============================================================
// CONFIGURATION
// ============================================================
define('DATA_FILE', __DIR__ . '/data.json');
define('CACHE_DURATION', 86400); // seconds — 24 h
define('MAX_HISTORY', 7); // days of snapshots
define('MAX_RETRIES', 3); // fetch retry attempts
define('ENGINE_VERSION', '1.1.0');
// ============================================================
// ROUTING
// ============================================================
$action = $_GET['action'] ?? '';
if ($action !== '') {
header('Content-Type: application/json; charset=utf-8');
header('X-Content-Type-Options: nosniff');
route_api($action);
exit;
}
// Render dashboard
render_dashboard();
exit;
// ============================================================
// API ROUTER
// ============================================================
function route_api(string $action): void
{
switch ($action) {
case 'get': api_get_value(); break;
case 'list': api_list_items(); break;
case 'refresh': api_refresh_item(); break;
case 'refresh_all': api_refresh_all(); break;
case 'test_xpath': api_test_xpath(); break;
case 'test_extract': api_test_extract(); break;
case 'save': api_save_item(); break;
case 'delete': api_delete_item(); break;
default: json_out(['error' => 'Unknown action: ' . $action]);
}
}
// ============================================================
// JSON HELPERS
// ============================================================
function json_out(array $data): void
{
echo json_encode($data, JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES);
}
function read_data(): array
{
if (!file_exists(DATA_FILE)) {
$default = ['items' => [], '_meta' => ['created' => date('c'), 'version' => ENGINE_VERSION]];
write_data($default);
return $default;
}
$raw = file_get_contents(DATA_FILE);
$data = json_decode($raw, true);
return is_array($data) ? $data : ['items' => []];
}
function write_data(array $data): bool
{
$data['_meta']['updated'] = date('c');
return (bool) file_put_contents(
DATA_FILE,
json_encode($data, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES),
LOCK_EX
);
}
// ============================================================
// NUMBER PARSER
// Handles: "NPR 1,33,456.78" → 133456.78
// "USD 133.45" → 133.45
// "Rs. 1,000" → 1000
// "$ 12,345.67" → 12345.67
// ============================================================
function parse_number(string $text): ?float
{
$text = trim($text);
// Strip currency prefixes / suffixes
$text = preg_replace(
'/^(NPR|USD|INR|EUR|GBP|AUD|CAD|JPY|CNY|Rs\.?|रू\.?|\$|€|£|¥|₹|₩|₺)\s*/iu',
'',
$text
);
$text = preg_replace(
'/\s*(NPR|USD|INR|EUR|GBP|AUD|CAD|JPY|CNY|Rs\.?)\s*$/iu',
'',
$text
);
// Remove thousands separators (commas)
$text = str_replace(',', '', $text);
$text = trim($text);
// Extract first numeric value (including negatives & decimals)
if (preg_match('/[-+]?\d+(?:\.\d+)?/', $text, $m)) {
return (float) $m[0];
}
return null;
}
// ============================================================
// HTTP FETCH WITH RETRY + EXPONENTIAL BACKOFF
// ============================================================
function fetch_url(string $url, int $max_retries = MAX_RETRIES): array
{
$last_error = '';
for ($attempt = 1; $attempt <= $max_retries; $attempt++) {
if ($attempt > 1) {
sleep((int) pow(2, $attempt - 1)); // 2s, 4s backoff
}
$ctx = stream_context_create([
'http' => [
'timeout' => 20,
'user_agent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0 Safari/537.36',
'follow_location' => true,
'max_redirects' => 5,
'ignore_errors' => true,
],
'ssl' => [
'verify_peer' => false,
'verify_peer_name' => false,
],
]);
$html = @file_get_contents($url, false, $ctx);
if ($html !== false && strlen($html) > 0) {
return ['success' => true, 'html' => $html];
}
$last_error = error_get_last()['message'] ?? 'No response';
}
return [
'success' => false,
'error' => "Fetch failed after {$max_retries} attempt(s): {$last_error}",
];
}
// ============================================================
// XPATH EXTRACTOR
// ============================================================
function extract_by_xpath(string $html, string $xpath_expr): array
{
$dom = new DOMDocument();
libxml_use_internal_errors(true);
$dom->loadHTML('<?xml encoding="utf-8"?>' . $html);
libxml_clear_errors();
$xp = new DOMXPath($dom);
$nodes = @$xp->query($xpath_expr);
if ($nodes === false) {
return ['success' => false, 'error' => 'Invalid XPath expression'];
}
if ($nodes->length === 0) {
return ['success' => false, 'error' => 'XPath matched 0 nodes — check selector'];
}
$node = $nodes->item(0);
$raw_text = trim($node->textContent ?? $node->nodeValue ?? '');
if ($raw_text === '') {
return ['success' => false, 'error' => 'XPath matched but node is empty (JS-rendered content?)'];
}
$value = parse_number($raw_text);
if ($value === null) {
return [
'success' => false,
'error' => "Could not parse number from: \"{$raw_text}\"",
'raw' => $raw_text,
];
}
return ['success' => true, 'raw' => $raw_text, 'value' => $value];
}
// ============================================================
// REGEX EXTRACTOR (NEW — for JS-rendered sites)
//
// Use case: Data is embedded in a <script> tag like:
// var rate = "133.45";
// {"usd": {"buy": 133.45, ...}}
//
// The regex MUST contain exactly one capture group.
// Example patterns:
// /"buying"\s*:\s*"?([\d,.]+)"?/
// /var\s+rate\s*=\s*"?([\d,.]+)/
// /"usd_buy"\s*:\s*([\d.]+)/
// ============================================================
function extract_by_regex(string $html, string $regex_pattern): array
{
// Validate the regex first
if (@preg_match($regex_pattern, '') === false) {
return ['success' => false, 'error' => 'Invalid regex pattern: ' . preg_last_error_msg()];
}
if (!preg_match($regex_pattern, $html, $m)) {
return ['success' => false, 'error' => 'Regex did not match anything in the page source'];
}
// Use first capture group, or full match if no group
$raw_text = trim($m[1] ?? $m[0]);
if ($raw_text === '') {
return ['success' => false, 'error' => 'Regex matched but capture group is empty'];
}
$value = parse_number($raw_text);
if ($value === null) {
return [
'success' => false,
'error' => "Could not parse number from regex match: \"{$raw_text}\"",
'raw' => $raw_text,
];
}
return ['success' => true, 'raw' => $raw_text, 'value' => $value];
}
// ============================================================
// JSON PATH EXTRACTOR (NEW — for API endpoints)
//
// Use case: The JS on ocr.gov.np calls an API like:
// https://ocr.gov.np/api/forex/rates
// which returns JSON. Instead of scraping the HTML,
// hit the API directly and extract with a dot-path.
//
// Example json_path: "data.0.rates.usd.buying"
// → $json['data'][0]['rates']['usd']['buying']
// ============================================================
function extract_by_json_path(string $json_str, string $json_path): array
{
$data = json_decode($json_str, true);
if ($data === null && json_last_error() !== JSON_ERROR_NONE) {
return ['success' => false, 'error' => 'Response is not valid JSON: ' . json_last_error_msg()];
}
$segments = preg_split('/\./', $json_path);
$current = $data;
foreach ($segments as $seg) {
$seg = trim($seg);
if ($seg === '') continue;
if (is_array($current)) {
// Try both string key and numeric index
if (array_key_exists($seg, $current)) {
$current = $current[$seg];
} elseif (is_numeric($seg) && array_key_exists((int) $seg, $current)) {
$current = $current[(int) $seg];
} else {
return [
'success' => false,
'error' => "JSON path segment \"{$seg}\" not found. Available keys: " .
implode(', ', array_slice(array_keys($current), 0, 15)),
];
}
} else {
return [
'success' => false,
'error' => "Cannot traverse into non-array at segment \"{$seg}\" (value is: " .
substr(var_export($current, true), 0, 60) . ")",
];
}
}
$raw_text = is_string($current) ? $current : (string) $current;
if (is_numeric($current)) {
return ['success' => true, 'raw' => $raw_text, 'value' => (float) $current];
}
$value = parse_number($raw_text);
if ($value === null) {
return [
'success' => false,
'error' => "Could not parse number from JSON value: \"{$raw_text}\"",
'raw' => $raw_text,
];
}
return ['success' => true, 'raw' => $raw_text, 'value' => $value];
}
// ============================================================
// EVALUATE — LOCATOR (UPDATED — supports method field)
//
// method: 'xpath' (default) — classic XPath on HTML
// 'regex' — regex on raw HTML source
// 'json_path' — fetch JSON API, traverse by path
// ============================================================
function eval_locator(array $item): array
{
$method = $item['method'] ?? 'xpath';
$fetch = fetch_url($item['url']);
if (!$fetch['success']) {
return ['success' => false, 'error' => 'ERROR: ' . $fetch['error']];
}
switch ($method) {
// ── REGEX: search raw HTML for a pattern ──
case 'regex':
$patterns = array_filter([
$item['regex'] ?? '',
$item['regex2'] ?? '',
$item['regex3'] ?? '',
]);
foreach ($patterns as $pat) {
$result = extract_by_regex($fetch['html'], $pat);
if ($result['success']) return $result;
}
return ['success' => false, 'error' => 'ERROR: All regex patterns failed'];
// ── JSON PATH: parse response as JSON ──
case 'json_path':
$paths = array_filter([
$item['json_path'] ?? '',
$item['json_path2'] ?? '',
]);
foreach ($paths as $jp) {
$result = extract_by_json_path($fetch['html'], $jp);
if ($result['success']) return $result;
}
return ['success' => false, 'error' => 'ERROR: All JSON path selectors failed'];
// ── XPATH (default): classic DOM extraction ──
case 'xpath':
default:
$xpaths = array_filter([
$item['xpath'] ?? '',
$item['xpath2'] ?? '',
$item['xpath3'] ?? '',
]);
foreach ($xpaths as $xp) {
$result = extract_by_xpath($fetch['html'], $xp);
if ($result['success']) return $result;
}
return ['success' => false, 'error' => 'ERROR: All XPath selectors failed'];
}
}
// ============================================================
// CIRCULAR DEPENDENCY DETECTION
// ============================================================
function has_circular_dep(array $data, string $item_name, string $formula, array $visited = []): bool
{
$visited[] = $item_name;
preg_match_all('/\[([a-zA-Z0-9_]+)\]/', $formula, $m);
foreach (($m[1] ?? []) as $dep) {
if (in_array($dep, $visited, true)) return true;
$dep_item = $data['items'][$dep] ?? null;
if ($dep_item && !empty($dep_item['formula'])) {
if (has_circular_dep($data, $dep, $dep_item['formula'], $visited)) return true;
}
}
return false;
}
// ============================================================
// SAFE MATH EVALUATOR
// Only allows: digits, operators, parens, decimals, whitespace
// ============================================================
function safe_eval_math(string $expr): ?float
{
$clean = preg_replace('/[^0-9+\-*\/.() \t]/', '', $expr);
$clean = trim($clean);
if ($clean === '' || !preg_match('/\d/', $clean)) return null;
try {
$result = null;
// phpcs:ignore Squiz.PHP.Eval.Discouraged
eval('$result = ' . $clean . ';');
return is_numeric($result) ? (float) $result : null;
} catch (\Throwable $e) {
return null;
}
}
// ============================================================
// SAFE BOOLEAN EVALUATOR
// ============================================================
function safe_eval_bool(string $expr): bool
{
$clean = preg_replace('/[^0-9+\-*\/.() \t<>=!&|]/', '', $expr);
$clean = trim($clean);
if ($clean === '') return false;
try {
$result = false;
eval('$result = (bool)(' . $clean . ');');
return (bool) $result;
} catch (\Throwable $e) {
return false;
}
}
// ============================================================
// EVALUATE — CALCULATOR
// ============================================================
function eval_calculator(array $item, array $data, int $depth = 0): array
{
if ($depth > 20) return ['success' => false, 'error' => 'ERROR: Max recursion depth exceeded'];
$formula = $item['formula'];
preg_match_all('/\[([a-zA-Z0-9_]+)\]/', $formula, $m);
foreach (($m[1] ?? []) as $dep) {
if (!isset($data['items'][$dep])) {
return ['success' => false, 'error' => "ERROR: Dependency '[{$dep}]' not found"];
}
$dep_val = get_item_value($data['items'][$dep], $data, $depth + 1);
if ($dep_val === null) {
return ['success' => false, 'error' => "ERROR: Could not resolve '[{$dep}]'"];
}
$formula = str_replace("[{$dep}]", (string) $dep_val, $formula);
}
$result = safe_eval_math($formula);
if ($result === null) {
return ['success' => false, 'error' => "ERROR: Formula evaluation failed: \"{$formula}\""];
}
return ['success' => true, 'value' => $result];
}
// ============================================================
// EVALUATE — FORMATTER (pipeline of operations)
// ============================================================
function eval_formatter(array $item, array $data, int $depth = 0): array
{
$src_name = $item['source'] ?? '';
if (!isset($data['items'][$src_name])) {
return ['success' => false, 'error' => "ERROR: Source item '{$src_name}' not found"];
}
$value = get_item_value($data['items'][$src_name], $data, $depth + 1);
if ($value === null) {
return ['success' => false, 'error' => "ERROR: Could not resolve source '{$src_name}'"];
}
foreach (($item['pipeline'] ?? []) as $step) {
[$op, $arg] = array_pad(explode(':', $step, 2), 2, '');
switch ($op) {
case 'round': $value = round((float) $value, (int) $arg); break;
case 'floor': $value = floor((float) $value); break;
case 'ceil': $value = ceil((float) $value); break;
case 'abs': $value = abs((float) $value); break;
case 'multiply': $value = (float) $value * (float) $arg; break;
case 'divide': $value = $arg != 0 ? (float)$value / (float)$arg : $value; break;
case 'prefix': $value = $arg . $value; break;
case 'suffix': $value = $value . $arg; break;
case 'number_format':
[$dec, $ds, $ts] = array_pad(explode(',', $arg, 3), 3, '');
$value = number_format((float) $value, (int)($dec ?: 2), $ds ?: '.', $ts ?: ',');
break;
}
}
return ['success' => true, 'value' => $value];
}
// ============================================================
// EVALUATE — SWITCH / CONDITIONAL
// ============================================================
function eval_switch(array $item, array $data, int $depth = 0): array
{
$condition = $item['condition'] ?? '';
$if_true = $item['if_true'] ?? '';
$if_false = $item['if_false'] ?? '';
// Resolve [item] references in condition
preg_match_all('/\[([a-zA-Z0-9_]+)\]/', $condition, $m);
foreach (($m[1] ?? []) as $dep) {
if (!isset($data['items'][$dep])) {
return ['success' => false, 'error' => "ERROR: Dependency '[{$dep}]' not found in condition"];
}
$dep_val = get_item_value($data['items'][$dep], $data, $depth + 1);
if ($dep_val === null) {
return ['success' => false, 'error' => "ERROR: Could not resolve '[{$dep}]'"];
}
$condition = str_replace("[{$dep}]", (string) $dep_val, $condition);
}
$branch = safe_eval_bool($condition) ? $if_true : $if_false;
$target = $data['items'][$branch] ?? null;
if (!$target) {
return ['success' => false, 'error' => "ERROR: Target item '{$branch}' not found"];
}
$value = get_item_value($target, $data, $depth + 1);
if ($value === null) {
return ['success' => false, 'error' => "ERROR: Could not resolve target '{$branch}'"];
}
return ['success' => true, 'value' => $value, 'branch' => $branch];
}
// ============================================================
// UNIFIED VALUE GETTER
// ============================================================
function get_item_value(array $item, array $data, int $depth = 0): mixed
{
if ($depth > 20) return null;
switch ($item['type']) {
case 'constant':
return isset($item['value']) ? (float) $item['value'] : null;
case 'locator':
return isset($item['last_value']) ? (float) $item['last_value'] : null;
case 'calculator':
$r = eval_calculator($item, $data, $depth);
return $r['success'] ? $r['value'] : null;
case 'formatter':
$r = eval_formatter($item, $data, $depth);
return $r['success'] ? $r['value'] : null;
case 'switch':
$r = eval_switch($item, $data, $depth);
return $r['success'] ? $r['value'] : null;
}
return null;
}
// ============================================================
// HISTORY SNAPSHOT
// ============================================================
function push_history(array &$item, float $value, string $date): void
{
if (!isset($item['history'])) $item['history'] = [];
$found = false;
foreach ($item['history'] as &$snap) {
if ($snap['date'] === $date) { $snap['value'] = $value; $found = true; break; }
}
unset($snap);
if (!$found) $item['history'][] = ['date' => $date, 'value' => $value];
// Sort newest-first, keep MAX_HISTORY
usort($item['history'], fn($a, $b) => strcmp($b['date'], $a['date']));
$item['history'] = array_slice($item['history'], 0, MAX_HISTORY);
}
// ============================================================
// CACHE CHECK
// ============================================================
function cache_valid(array $item): bool
{
if (empty($item['last_updated'])) return false;
return (time() - strtotime($item['last_updated'])) < CACHE_DURATION;
}
// ============================================================
// REFRESH SINGLE ITEM (mutates $data by reference)
// ============================================================
function refresh_item(string $name, array &$data): array
{
if (!isset($data['items'][$name])) {
return ['success' => false, 'error' => "Item '{$name}' not found"];
}
$item = &$data['items'][$name];
$now = date('c');
$date = date('Y-m-d');
$set_ok = function(float $value) use (&$item, $now, $date): void {
$item['last_value'] = $value;
$item['status'] = 'ok';
$item['last_updated'] = $now;
$item['error'] = null;
push_history($item, $value, $date);
};
$set_err = function(string $error) use (&$item, $now): void {
$item['status'] = 'error';
$item['error'] = $error;
$item['last_updated'] = $now;
};
switch ($item['type']) {
case 'locator':
$r = eval_locator($item);
$r['success'] ? $set_ok($r['value']) : $set_err($r['error']);
if ($r['success']) $item['last_raw'] = $r['raw'];
break;
case 'calculator':
$r = eval_calculator($item, $data);
$r['success'] ? $set_ok($r['value']) : $set_err($r['error']);
break;
case 'constant':
$item['last_value'] = (float) ($item['value'] ?? 0);
$item['status'] = 'ok';
$item['last_updated'] = $now;
push_history($item, $item['last_value'], $date);
break;
case 'formatter':
$r = eval_formatter($item, $data);
$r['success'] ? $set_ok(is_numeric($r['value']) ? (float)$r['value'] : 0) : $set_err($r['error']);
// Store as string for formatters (may have prefix/suffix)
if ($r['success']) $item['display_value'] = $r['value'];
break;
case 'switch':
$r = eval_switch($item, $data);
$r['success'] ? $set_ok($r['value']) : $set_err($r['error']);
if ($r['success'] && isset($r['branch'])) $item['last_branch'] = $r['branch'];
break;
}
return ['success' => $item['status'] === 'ok', 'item' => $item];
}
// ============================================================
// API — GET VALUE (used by WordPress shortcode)
// ============================================================
function api_get_value(): void
{
$name = trim($_GET['name'] ?? '');
$format = trim($_GET['format'] ?? 'formatted');
$fallback = trim($_GET['fallback'] ?? 'N/A');
$decimals = max(0, (int) ($_GET['decimals'] ?? 2));
if ($name === '') { json_out(['error' => 'name parameter required']); return; }
$data = read_data();
if (!isset($data['items'][$name])) {
header('Content-Type: text/plain; charset=utf-8');
echo $fallback;
return;
}
$item = &$data['items'][$name];
// Auto-refresh if cache expired
if (!cache_valid($item)) {
refresh_item($name, $data);
write_data($data);
$item = $data['items'][$name];
}
header('Content-Type: text/plain; charset=utf-8');
// Formatter items may have a display_value (string with prefix/suffix)
if ($item['type'] === 'formatter' && isset($item['display_value'])) {
echo $item['display_value'];
return;
}
$value = $item['last_value'] ?? null;
if ($value === null) { echo $fallback; return; }
switch ($format) {
case 'raw': echo $value; break;
case 'integer': echo (int) round((float) $value); break;
default: echo number_format((float) $value, $decimals); break;
}
}
// ============================================================
// API — LIST ALL ITEMS
// ============================================================
function api_list_items(): void
{
$data = read_data();
json_out(['items' => $data['items'], 'count' => count($data['items'])]);
}
// ============================================================
// API — REFRESH SINGLE
// ============================================================
function api_refresh_item(): void
{
$name = trim($_GET['name'] ?? '');
if ($name === '') { json_out(['error' => 'name required']); return; }
$data = read_data();
$result = refresh_item($name, $data);
write_data($data);
json_out($result);
}
// ============================================================
// API — REFRESH ALL
// Returns results indexed by item name
// Evaluates in dependency order: constants → locators → calcs → formatters → switches
// ============================================================
function api_refresh_all(): void
{
$data = read_data();
$results = [];
$order = ['constant', 'locator', 'calculator', 'formatter', 'switch'];
// Build ordered list
$sorted = [];
foreach ($order as $type) {
foreach ($data['items'] as $name => $item) {
if ($item['type'] === $type) $sorted[$name] = $item;
}
}
foreach (array_keys($sorted) as $name) {
$r = refresh_item($name, $data);
$results[$name] = [
'success' => $r['success'],
'status' => $data['items'][$name]['status'],
'value' => $data['items'][$name]['last_value'] ?? null,
'error' => $data['items'][$name]['error'] ?? null,
];
}
write_data($data);
json_out(['results' => $results, 'total' => count($results)]);
}
// ============================================================
// API — TEST XPATH (live tester, no save)
// ============================================================
function api_test_xpath(): void
{
$input = json_decode(file_get_contents('php://input'), true) ?? [];
$url = trim($input['url'] ?? $_POST['url'] ?? '');
$xpath = trim($input['xpath'] ?? $_POST['xpath'] ?? '');
if ($url === '' || $xpath === '') {
json_out(['error' => 'Both url and xpath are required']);
return;
}
$fetch = fetch_url($url, 1);
if (!$fetch['success']) {
json_out(['error' => 'Failed to fetch URL: ' . $fetch['error']]);
return;
}
json_out(extract_by_xpath($fetch['html'], $xpath));
}
// ============================================================
// API — TEST EXTRACT (NEW — universal live tester)
// Supports method: xpath, regex, json_path
// ============================================================
function api_test_extract(): void
{
$input = json_decode(file_get_contents('php://input'), true) ?? [];
$url = trim($input['url'] ?? '');
$method = trim($input['method'] ?? 'xpath');
$expr = trim($input['expr'] ?? '');
if ($url === '' || $expr === '') {
json_out(['error' => 'Both url and expression are required']);
return;
}
$fetch = fetch_url($url, 1);
if (!$fetch['success']) {
json_out(['error' => 'Failed to fetch URL: ' . $fetch['error']]);
return;
}
switch ($method) {
case 'regex':
json_out(extract_by_regex($fetch['html'], $expr));
break;
case 'json_path':
json_out(extract_by_json_path($fetch['html'], $expr));
break;
case 'xpath':
default:
json_out(extract_by_xpath($fetch['html'], $expr));
break;
}
}
// ============================================================
// API — SAVE ITEM (create or update)
// ============================================================
function api_save_item(): void
{
$input = json_decode(file_get_contents('php://input'), true) ?? [];
$raw_name = trim($input['name'] ?? '');
if ($raw_name === '') { json_out(['error' => 'name is required']); return; }
// Sanitise name
$name = preg_replace('/[^a-zA-Z0-9_]/', '_', $raw_name);
$type = $input['type'] ?? 'locator';
$data = read_data();
$is_new = !isset($data['items'][$name]);
// Preserve existing metadata if editing
$item = $data['items'][$name] ?? [];
$item['type'] = $type;
if (!empty($input['label'])) $item['label'] = trim($input['label']);
if (!empty($input['description'])) $item['description'] = trim($input['description']);
switch ($type) {
case 'locator':
$url = trim($input['url'] ?? '');
$method = trim($input['method'] ?? 'xpath');
if ($url === '') {
json_out(['error' => 'url is required for locators']);
return;
}
$item['url'] = $url;
$item['method'] = $method;
// Clear all method-specific fields first
unset($item['xpath'], $item['xpath2'], $item['xpath3']);
unset($item['regex'], $item['regex2'], $item['regex3']);
unset($item['json_path'], $item['json_path2']);
switch ($method) {
case 'regex':
$regex = trim($input['regex'] ?? '');
if ($regex === '') {
json_out(['error' => 'regex pattern is required']);
return;
}
$item['regex'] = $regex;
$item['regex2'] = trim($input['regex2'] ?? '');
$item['regex3'] = trim($input['regex3'] ?? '');
break;
case 'json_path':
$jp = trim($input['json_path'] ?? '');
if ($jp === '') {
json_out(['error' => 'json_path is required']);
return;
}
$item['json_path'] = $jp;
$item['json_path2'] = trim($input['json_path2'] ?? '');
break;
case 'xpath':
default:
$xpath = trim($input['xpath'] ?? '');
if ($xpath === '') {
json_out(['error' => 'xpath is required for xpath method']);
return;
}
$item['xpath'] = $xpath;
$item['xpath2'] = trim($input['xpath2'] ?? '');
$item['xpath3'] = trim($input['xpath3'] ?? '');
break;
}
break;
case 'calculator':
$formula = trim($input['formula'] ?? '');
if ($formula === '') { json_out(['error' => 'formula is required']); return; }
if (has_circular_dep($data, $name, $formula)) {
json_out(['error' => 'ERROR: Circular reference detected in formula']);
return;
}
$item['formula'] = $formula;
break;
case 'constant':
$val = $input['value'] ?? '';
if ($val === '') { json_out(['error' => 'value is required for constants']); return; }
$item['value'] = (float) $val;
$item['last_value'] = (float) $val;
$item['status'] = 'ok';
$item['last_updated'] = date('c');
push_history($item, (float) $val, date('Y-m-d'));
break;
case 'formatter':
$source = trim($input['source'] ?? '');
if ($source === '') { json_out(['error' => 'source is required for formatters']); return; }
if (!isset($data['items'][$source])) {
json_out(['error' => "Source item '{$source}' does not exist"]);
return;
}
$item['source'] = $source;
$item['pipeline'] = $input['pipeline'] ?? [];
break;
case 'switch':
$cond = trim($input['condition'] ?? '');
$if_true = trim($input['if_true'] ?? '');
$if_false = trim($input['if_false'] ?? '');
if ($cond === '' || $if_true === '' || $if_false === '') {
json_out(['error' => 'condition, if_true and if_false are all required']);
return;
}
$item['condition'] = $cond;
$item['if_true'] = $if_true;
$item['if_false'] = $if_false;
break;
default:
json_out(['error' => "Unknown type: {$type}"]);
return;
}
$data['items'][$name] = $item;
write_data($data);
json_out(['success' => true, 'name' => $name, 'is_new' => $is_new]);
}
// ============================================================
// API — DELETE ITEM
// ============================================================
function api_delete_item(): void
{
$name = trim($_GET['name'] ?? '');
if ($name === '') { json_out(['error' => 'name required']); return; }
$data = read_data();
if (!isset($data['items'][$name])) {
json_out(['error' => "Item '{$name}' not found"]);
return;
}
unset($data['items'][$name]);
write_data($data);
json_out(['success' => true, 'deleted' => $name]);
}
// ============================================================
// SPARKLINE SVG GENERATOR (fixed version)
// ============================================================
function render_sparkline_fixed(array $history): string
{
if (count($history) < 2) {
return '<span style="color:var(--c-muted);font-size:11px">—</span>';
}
usort($history, fn($a, $b) => strcmp($a['date'], $b['date']));
$values = array_column($history, 'value');
$min = min($values); $max = max($values);
$range = $max - $min ?: 1;
$n = count($values);
$W = 72; $H = 26; $pad = 3;
$coords = [];
foreach ($values as $i => $v) {
$x = $pad + ($i / max($n - 1, 1)) * ($W - $pad * 2);
$y = $H - $pad - (($v - $min) / $range) * ($H - $pad * 2);
$coords[] = [round($x, 1), round($y, 1)];
}
$poly_pts = implode(' ', array_map(fn($c) => "{$c[0]},{$c[1]}", $coords));
$last = end($values); $first = reset($values);
$color = $last > $first ? '#22c55e' : ($last < $first ? '#ef4444' : '#94a3b8');
$lx = end($coords)[0]; $ly = end($coords)[1];
$fx = $coords[0][0];
$grad_id = 'sg_' . crc32(json_encode($values));
$area = "M{$coords[0][0]},{$coords[0][1]} " .
implode(' ', array_map(fn($c) => "L{$c[0]},{$c[1]}", array_slice($coords, 1))) .
" L{$lx},{$H} L{$fx},{$H} Z";
return "<svg width='{$W}' height='{$H}' viewBox='0 0 {$W} {$H}' xmlns='http://www.w3.org/2000/svg' style='display:block'>
<defs><linearGradient id='{$grad_id}' x1='0' y1='0' x2='0' y2='1'>
<stop offset='0%' stop-color='{$color}' stop-opacity='0.2'/>
<stop offset='100%' stop-color='{$color}' stop-opacity='0'/>
</linearGradient></defs>
<path d='{$area}' fill='url(#{$grad_id})'/>
<polyline points='{$poly_pts}' fill='none' stroke='{$color}' stroke-width='1.6' stroke-linejoin='round' stroke-linecap='round'/>
<circle cx='{$lx}' cy='{$ly}' r='2.2' fill='{$color}'/>
</svg>";
}
// ============================================================
// TIME AGO HELPER
// ============================================================
function time_ago(string $ts): string
{
$t = strtotime($ts);
if (!$t) return '—';
$d = time() - $t;
if ($d < 60) return 'just now';
if ($d < 3600) return floor($d / 60) . 'm ago';
if ($d < 86400) return floor($d / 3600) . 'h ago';
return floor($d / 86400) . 'd ago';
}
// ============================================================
// DASHBOARD RENDER
// ============================================================
function render_dashboard(): void
{
$data = read_data();
$items = $data['items'];
$total = count($items);
$ok_ct = count(array_filter($items, fn($i) => ($i['status'] ?? '') === 'ok'));
$err_ct = count(array_filter($items, fn($i) => ($i['status'] ?? '') === 'error'));
$pending_ct = $total - $ok_ct - $err_ct;
$type_counts = [];
foreach (['locator','calculator','constant','formatter','switch'] as $t) {
$type_counts[$t] = count(array_filter($items, fn($i) => $i['type'] === $t));
}
$all_items_json = json_encode($items, JSON_UNESCAPED_UNICODE | JSON_HEX_TAG | JSON_HEX_APOS | JSON_HEX_QUOT);
?>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>DataEngine — Dashboard</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=DM+Sans:ital,opsz,wght@0,9..40,300;0,9..40,400;0,9..40,500;0,9..40,600;1,9..40,400&family=JetBrains+Mono:wght@400;500&display=swap">
<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Material+Symbols+Outlined:opsz,wght,FILL,GRAD@20,300,0,0">
<style>
/* ═══════════════════════════════════════════════════════
RESET & TOKENS
═══════════════════════════════════════════════════════ */
*,*::before,*::after{box-sizing:border-box;margin:0;padding:0}
:root{
--bg: #ffffff;
--bg-2: #f7f6f3;
--bg-3: #efeeeb;
--bg-hover: #f1f0ed;
--border: #e5e4e0;
--border-2: #d3d2cd;
--c-text: #37352f;
--c-sub: #787774;
--c-muted: #acaba7;
--accent: #2383e2;
--accent-light: #e8f1fd;
--accent-hover: #1a6bc9;
--ok: #16a34a; --ok-bg: #dcfce7;
--warn: #ca8a04; --warn-bg: #fef9c3;
--err: #dc2626; --err-bg: #fee2e2;
--cached: #7c3aed; --cached-bg:#ede9fe;
--loc-c:#2383e2; --loc-bg:#e8f1fd;
--calc-c:#d97706;--calc-bg:#fef3c7;
--const-c:#16a34a;--const-bg:#dcfce7;
--fmt-c:#7c3aed; --fmt-bg:#ede9fe;
--sw-c:#db2777; --sw-bg:#fce7f3;
--sidebar-w:220px;
--topbar-h:56px;
--radius:6px;
--radius-lg:10px;
--shadow:0 1px 3px rgba(0,0,0,.07),0 1px 2px rgba(0,0,0,.04);
--shadow-lg:0 8px 24px rgba(0,0,0,.12),0 2px 8px rgba(0,0,0,.06);
--font:'DM Sans',-apple-system,BlinkMacSystemFont,sans-serif;
--mono:'JetBrains Mono',ui-monospace,'Fira Code',monospace;
}
html{font-size:14px;-webkit-font-smoothing:antialiased}
body{font-family:var(--font);background:var(--bg);color:var(--c-text);min-height:100vh;line-height:1.5}
.layout{display:flex;min-height:100vh}
.sidebar{
width:var(--sidebar-w);background:var(--bg-2);border-right:1px solid var(--border);
position:fixed;top:0;left:0;bottom:0;display:flex;flex-direction:column;z-index:100;
transition:transform .22s cubic-bezier(.4,0,.2,1);
}
.sidebar-inner{display:flex;flex-direction:column;height:100%;overflow-y:auto}
.sb-brand{padding:16px 16px 12px;display:flex;align-items:center;gap:9px;border-bottom:1px solid var(--border);font-size:15px;font-weight:600;color:var(--c-text);user-select:none}
.sb-brand .icon{font-size:20px;color:var(--accent)}
.sb-version{font-size:10px;color:var(--c-muted);font-weight:400;margin-left:auto;font-family:var(--mono)}
.sb-section-label{font-size:10.5px;font-weight:600;color:var(--c-muted);text-transform:uppercase;letter-spacing:.08em;padding:14px 14px 4px}
.sb-nav{padding:6px 8px;flex:1}
.nav-item{display:flex;align-items:center;gap:8px;padding:6px 8px;border-radius:var(--radius);cursor:pointer;color:var(--c-sub);font-size:13px;transition:background .12s,color .12s;user-select:none}
.nav-item:hover{background:var(--bg-hover)}
.nav-item.active{background:var(--bg-3);color:var(--c-text);font-weight:500}
.nav-item .ms{font-size:17px}
.nav-count{margin-left:auto;background:var(--bg-3);color:var(--c-sub);border-radius:20px;padding:0 6px;font-size:10.5px;font-weight:600}
.sb-divider{height:1px;background:var(--border);margin:8px 12px}
.main{margin-left:var(--sidebar-w);flex:1;display:flex;flex-direction:column}
.topbar{height:var(--topbar-h);display:flex;align-items:center;gap:10px;padding:0 20px;border-bottom:1px solid var(--border);position:sticky;top:0;background:rgba(255,255,255,.92);backdrop-filter:blur(8px);z-index:50}
.topbar-title{font-size:14px;font-weight:600}
.search-box{display:flex;align-items:center;gap:7px;background:var(--bg-2);border:1px solid var(--border);border-radius:var(--radius);padding:5px 10px;min-width:200px;transition:border-color .15s,box-shadow .15s}
.search-box:focus-within{border-color:var(--accent);box-shadow:0 0 0 3px rgba(35,131,226,.12)}
.search-box .ms{font-size:16px;color:var(--c-muted)}
.search-box input{border:none;background:none;outline:none;font-family:var(--font);font-size:13px;color:var(--c-text);width:100%}
.search-box input::placeholder{color:var(--c-muted)}
.spacer{flex:1}
.btn{display:inline-flex;align-items:center;gap:6px;padding:6px 12px;border-radius:var(--radius);border:1px solid var(--border);background:var(--bg);color:var(--c-text);font-family:var(--font);font-size:13px;font-weight:500;cursor:pointer;white-space:nowrap;transition:background .12s,border-color .12s,box-shadow .12s;text-decoration:none}
.btn:hover{background:var(--bg-2)}
.btn:active{transform:translateY(1px)}
.btn .ms{font-size:15px}
.btn-primary{background:var(--accent);color:#fff;border-color:var(--accent)}
.btn-primary:hover{background:var(--accent-hover);border-color:var(--accent-hover)}
.btn-sm{padding:4px 9px;font-size:12px}
.btn-sm .ms{font-size:13px}
.btn-danger{background:var(--err-bg);color:var(--err);border-color:#fca5a5}
.btn-danger:hover{background:#fecaca}
.btn-icon{display:inline-flex;align-items:center;justify-content:center;padding:5px;border-radius:var(--radius);border:none;background:none;cursor:pointer;color:var(--c-sub);transition:background .12s,color .12s}
.btn-icon:hover{background:var(--bg-hover);color:var(--c-text)}
.btn-icon .ms{font-size:16px}
.ms{font-family:'Material Symbols Outlined';font-style:normal;font-weight:normal;font-size:20px;line-height:1;display:inline-block;vertical-align:middle;letter-spacing:normal;text-transform:none;-webkit-font-smoothing:antialiased}
.btn-icon.danger{color:var(--err)}
.btn-icon.danger:hover{background:var(--err-bg)}
.content{padding:22px 20px;flex:1}
.stats-grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(130px,1fr));gap:10px;margin-bottom:20px}
.stat-card{background:var(--bg-2);border:1px solid var(--border);border-radius:var(--radius-lg);padding:14px 16px}
.stat-label{font-size:11px;font-weight:600;color:var(--c-muted);text-transform:uppercase;letter-spacing:.07em;margin-bottom:4px}
.stat-val{font-size:24px;font-weight:600}
.stat-val.c-ok{color:var(--ok)}
.stat-val.c-err{color:var(--err)}
.stat-val.c-cached{color:var(--cached)}
.tbl-wrap{border:1px solid var(--border);border-radius:var(--radius-lg);overflow:hidden;overflow-x:auto}
table{width:100%;border-collapse:collapse;min-width:760px}
thead{background:var(--bg-2);border-bottom:1px solid var(--border)}
th{padding:9px 14px;text-align:left;font-size:11px;font-weight:600;color:var(--c-sub);text-transform:uppercase;letter-spacing:.07em;white-space:nowrap}
td{padding:10px 14px;border-bottom:1px solid var(--border);vertical-align:middle}
tr:last-child td{border-bottom:none}
tbody tr{transition:background .1s}
tbody tr:hover{background:var(--bg-2)}
.item-name{font-weight:500;font-size:13.5px}
.item-sublabel{font-size:11px;color:var(--c-sub);margin-top:1px}
.item-method{font-size:10px;color:var(--c-muted);font-family:var(--mono);margin-top:1px}
.badge{display:inline-flex;align-items:center;gap:3px;padding:2px 8px;border-radius:20px;font-size:11.5px;font-weight:500}
.badge-locator{background:var(--loc-bg);color:var(--loc-c)}
.badge-calculator{background:var(--calc-bg);color:var(--calc-c)}
.badge-constant{background:var(--const-bg);color:var(--const-c)}
.badge-formatter{background:var(--fmt-bg);color:var(--fmt-c)}
.badge-switch{background:var(--sw-bg);color:var(--sw-c)}
.status-pill{display:inline-flex;align-items:center;gap:5px;padding:3px 9px;border-radius:20px;font-size:11.5px;font-weight:500}
.status-ok{background:var(--ok-bg);color:var(--ok)}
.status-error{background:var(--err-bg);color:var(--err)}
.status-pending{background:var(--bg-3);color:var(--c-sub)}
.status-cached{background:var(--cached-bg);color:var(--cached)}
.dot{width:6px;height:6px;border-radius:50%;flex-shrink:0}
.dot-ok{background:var(--ok)}
.dot-error{background:var(--err)}
.dot-pending{background:var(--c-muted)}
.dot-cached{background:var(--cached)}
.val-cell{font-family:var(--mono);font-size:12.5px;font-weight:500}
.val-null{color:var(--c-muted)}
.val-error{color:var(--err);font-size:11px;font-family:var(--mono)}
.sc-chip{display:inline-flex;align-items:center;gap:4px;background:var(--bg-2);border:1px solid var(--border);border-radius:var(--radius);padding:3px 8px;font-family:var(--mono);font-size:11px;color:var(--c-sub);cursor:pointer;white-space:nowrap;transition:background .12s,color .12s,border-color .12s}
.sc-chip:hover{background:var(--accent-light);border-color:var(--accent);color:var(--accent)}
.sc-chip .ms{font-size:11px}
.actions{display:flex;align-items:center;gap:1px}
.time-cell{font-size:11.5px;color:var(--c-sub);white-space:nowrap}
.spark-cell{display:flex;align-items:center}
.empty-state{text-align:center;padding:60px 20px;color:var(--c-sub)}
.empty-state .ms{font-size:44px;color:var(--c-muted);display:block;margin-bottom:12px}
.empty-state p{font-size:14px}
/* SLIDE-IN MODAL */
.modal-overlay{position:fixed;inset:0;background:rgba(0,0,0,.3);z-index:200;display:none;align-items:flex-start;justify-content:flex-end;backdrop-filter:blur(2px)}
.modal-overlay.open{display:flex}
.modal-panel{width:min(520px,100vw);height:100vh;background:var(--bg);border-left:1px solid var(--border);box-shadow:var(--shadow-lg);display:flex;flex-direction:column;overflow:hidden;animation:slideIn .2s cubic-bezier(.4,0,.2,1)}
@keyframes slideIn{from{transform:translateX(100%)}to{transform:translateX(0)}}
.modal-hdr{padding:16px 18px;border-bottom:1px solid var(--border);display:flex;align-items:center;gap:10px;flex-shrink:0}
.modal-title{font-size:14px;font-weight:600;flex:1}
.modal-body{flex:1;overflow-y:auto;padding:18px}
.modal-ftr{padding:12px 18px;border-top:1px solid var(--border);display:flex;gap:8px;justify-content:flex-end;flex-shrink:0}
/* FORM */
.fg{margin-bottom:14px}
.fg-row{display:grid;grid-template-columns:1fr 1fr;gap:10px}
.flabel{display:block;font-size:12px;font-weight:500;color:var(--c-text);margin-bottom:5px}
.flabel .req{color:var(--err);margin-left:2px}
.fhint{font-size:11px;color:var(--c-sub);margin-top:4px}
.ferr{font-size:11px;color:var(--err);margin-top:4px}
.finput,.fselect{width:100%;padding:7px 10px;border:1px solid var(--border);border-radius:var(--radius);font-family:var(--font);font-size:13px;color:var(--c-text);background:var(--bg);outline:none;transition:border-color .15s,box-shadow .15s}
.finput:focus,.fselect:focus{border-color:var(--accent);box-shadow:0 0 0 3px rgba(35,131,226,.12)}
.finput::placeholder{color:var(--c-muted)}
.finput.mono{font-family:var(--mono);font-size:12px}
.fselect{appearance:none;cursor:pointer;background-image:url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='16' viewBox='0 0 24 24'%3E%3Cpath fill='%23787774' d='M7 10l5 5 5-5z'/%3E%3C/svg%3E");background-repeat:no-repeat;background-position:right 8px center;padding-right:30px}
.form-section{font-size:11px;font-weight:600;color:var(--c-muted);text-transform:uppercase;letter-spacing:.08em;margin:16px 0 10px;padding-bottom:6px;border-bottom:1px solid var(--border)}
/* METHOD TABS */
.method-tabs{display:flex;gap:4px;margin-bottom:14px}
.method-tab{flex:1;padding:8px 10px;border:1px solid var(--border);border-radius:var(--radius);background:var(--bg);cursor:pointer;text-align:center;font-size:12px;font-weight:500;color:var(--c-sub);transition:all .15s}
.method-tab:hover{background:var(--bg-2)}
.method-tab.active{background:var(--accent-light);border-color:var(--accent);color:var(--accent)}
.method-tab .ms{font-size:14px;display:block;margin:0 auto 2px}
.method-tab small{display:block;font-size:10px;font-weight:400;color:var(--c-muted);margin-top:1px}
.method-tab.active small{color:var(--accent)}
/* TESTER */
.tester-wrap{background:var(--bg-2);border:1px solid var(--border);border-radius:var(--radius-lg);margin-top:10px;overflow:hidden}
.tester-hdr{padding:9px 12px;display:flex;align-items:center;gap:7px;cursor:pointer;font-size:12.5px;font-weight:500;transition:background .12s;user-select:none}
.tester-hdr:hover{background:var(--bg-3)}
.tester-hdr .ms{font-size:15px;color:var(--accent)}
.tester-body{padding:12px;border-top:1px solid var(--border)}
.tester-result{margin-top:10px;padding:10px 12px;border-radius:var(--radius);font-family:var(--mono);font-size:11.5px;white-space:pre-wrap;word-break:break-all}
.tester-ok{background:var(--ok-bg);color:var(--ok);border:1px solid #86efac}
.tester-err{background:var(--err-bg);color:var(--err);border:1px solid #fca5a5}
/* INFO BOX */
.info-box{background:var(--accent-light);border:1px solid #bfdbfe;border-radius:var(--radius);padding:10px 12px;font-size:11.5px;color:#1e40af;margin-bottom:12px;line-height:1.55}
.info-box code{font-family:var(--mono);font-size:10.5px;background:rgba(0,0,0,.06);padding:1px 4px;border-radius:3px}
/* PIPELINE */
.pipeline-step{display:flex;align-items:center;gap:6px;margin-bottom:6px}
.pipeline-step .fselect{flex:0 0 120px}
.pipeline-step .finput{flex:1}
.deps-wrap{display:flex;flex-wrap:wrap;gap:5px;margin-top:6px}
.dep-chip{background:var(--accent-light);color:var(--accent);border-radius:var(--radius);padding:2px 8px;font-size:11.5px;font-weight:500;font-family:var(--mono)}
/* TOAST */
.toast-wrap{position:fixed;bottom:20px;right:20px;z-index:999;display:flex;flex-direction:column;gap:7px;pointer-events:none}
.toast{background:var(--c-text);color:#fff;padding:10px 14px;border-radius:var(--radius-lg);font-size:13px;font-weight:500;box-shadow:var(--shadow-lg);animation:toastIn .18s ease;display:flex;align-items:center;gap:8px;min-width:200px;max-width:320px;pointer-events:auto}
.toast.ok{background:var(--ok)}
.toast.err{background:var(--err)}
.toast .ms{font-size:15px}
@keyframes toastIn{from{opacity:0;transform:translateY(8px) scale(.97)}to{opacity:1;transform:translateY(0) scale(1)}}
.spin{display:inline-block;width:13px;height:13px;border:2px solid rgba(255,255,255,.3);border-top-color:#fff;border-radius:50%;animation:rotate .55s linear infinite;vertical-align:middle}
.spin.dark{border-color:var(--border);border-top-color:var(--accent)}
@keyframes rotate{to{transform:rotate(360deg)}}
.sb-overlay{display:none;position:fixed;inset:0;background:rgba(0,0,0,.25);z-index:90}
@media(max-width:768px){
.sidebar{transform:translateX(-100%)}
.sidebar.open{transform:translateX(0)}
.sb-overlay.open{display:block}
.main{margin-left:0}
.topbar{padding:0 12px}
.content{padding:14px 12px}
.search-box{flex:1;min-width:0}
.topbar-title,.btn-hide-mobile{display:none!important}
.stats-grid{grid-template-columns:1fr 1fr}
.modal-panel{width:100vw}
.hamburger{display:inline-flex!important}
}
@media(min-width:769px){.hamburger{display:none!important}}
.mt8{margin-top:8px}
</style>
</head>
<body>
<div class="sb-overlay" id="sbOverlay" onclick="closeSidebar()"></div>
<div class="layout">
<aside class="sidebar" id="sidebar">
<div class="sidebar-inner">
<div class="sb-brand">
<span class="ms icon">bolt</span>
DataEngine
<span class="sb-version">v<?= ENGINE_VERSION ?></span>
</div>
<nav class="sb-nav">
<div class="sb-section-label">Filter by Type</div>
<div class="nav-item active" id="nav-all" onclick="navFilter('all','')">
<span class="ms">grid_view</span>All Items
<span class="nav-count" id="nc-all"><?= $total ?></span>
</div>
<div class="nav-item" id="nav-locator" onclick="navFilter('locator','type')">
<span class="ms">travel_explore</span>Locators
<span class="nav-count" id="nc-locator"><?= $type_counts['locator'] ?></span>
</div>
<div class="nav-item" id="nav-calculator" onclick="navFilter('calculator','type')">
<span class="ms">calculate</span>Calculators
<span class="nav-count" id="nc-calculator"><?= $type_counts['calculator'] ?></span>
</div>
<div class="nav-item" id="nav-constant" onclick="navFilter('constant','type')">
<span class="ms">pin</span>Constants
<span class="nav-count" id="nc-constant"><?= $type_counts['constant'] ?></span>
</div>
<div class="nav-item" id="nav-formatter" onclick="navFilter('formatter','type')">
<span class="ms">text_format</span>Formatters
<span class="nav-count" id="nc-formatter"><?= $type_counts['formatter'] ?></span>
</div>
<div class="nav-item" id="nav-switch" onclick="navFilter('switch','type')">
<span class="ms">alt_route</span>Switches
<span class="nav-count" id="nc-switch"><?= $type_counts['switch'] ?></span>
</div>
<div class="sb-divider mt8"></div>
<div class="sb-section-label">Status</div>
<div class="nav-item" id="nav-ok" onclick="navFilter('ok','status')">
<span class="ms" style="color:var(--ok)">check_circle</span>Healthy
<span class="nav-count" id="nc-ok"><?= $ok_ct ?></span>
</div>
<div class="nav-item" id="nav-error" onclick="navFilter('error','status')">
<span class="ms" style="color:var(--err)">error</span>Errors
<span class="nav-count" id="nc-error"><?= $err_ct ?></span>
</div>
<div class="nav-item" id="nav-pending" onclick="navFilter('pending','status')">
<span class="ms" style="color:var(--c-muted)">schedule</span>Pending
<span class="nav-count" id="nc-pending"><?= $pending_ct ?></span>
</div>
</nav>
</div>
</aside>
<div class="main">
<header class="topbar">
<button class="btn-icon hamburger" onclick="toggleSidebar()"><span class="ms">menu</span></button>
<span class="topbar-title btn-hide-mobile">Dashboard</span>
<div class="search-box">
<span class="ms">search</span>
<input type="text" id="searchInput" placeholder="Search items…" oninput="doSearch(this.value)">
</div>
<div class="spacer"></div>
<button class="btn btn-hide-mobile" onclick="doRefreshAll()" id="btnRefreshAll">
<span class="ms">refresh</span>Refresh All
</button>
<button class="btn btn-primary" onclick="openAdd()">
<span class="ms">add</span><span class="btn-hide-mobile">Add Item</span>
</button>
</header>
<div class="content">
<div class="stats-grid">
<div class="stat-card">
<div class="stat-label">Total Items</div>
<div class="stat-val"><?= $total ?></div>
</div>
<div class="stat-card">
<div class="stat-label">Healthy</div>
<div class="stat-val c-ok"><?= $ok_ct ?></div>
</div>
<div class="stat-card">
<div class="stat-label">Errors</div>
<div class="stat-val c-err"><?= $err_ct ?></div>
</div>
<div class="stat-card">
<div class="stat-label">Pending</div>
<div class="stat-val" style="color:var(--c-sub)"><?= $pending_ct ?></div>
</div>
</div>
<div class="tbl-wrap">
<table id="mainTable">
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Value</th>
<th>Trend (7d)</th>
<th>Status</th>
<th>Updated</th>
<th>Shortcode</th>
<th></th>
</tr>
</thead>
<tbody id="tableBody">
<?php if (empty($items)): ?>
<tr>
<td colspan="8">
<div class="empty-state">
<span class="ms">inbox</span>
<p>No items yet. Click <strong>+ Add Item</strong> to get started.</p>
</div>
</td>
</tr>
<?php else: foreach ($items as $name => $item):
$status = $item['status'] ?? 'pending';
$value = $item['last_value'] ?? null;
$disp = $item['display_value'] ?? null;
$updated = $item['last_updated'] ?? null;
$history = $item['history'] ?? [];
$errMsg = $item['error'] ?? '';
$label = $item['label'] ?? '';
$type = $item['type'];
$method = $item['method'] ?? '';
$display_value = $disp !== null ? $disp : ($value !== null ? $value : null);
?>
<tr class="irow"
data-name="<?= htmlspecialchars($name) ?>"
data-type="<?= htmlspecialchars($type) ?>"
data-status="<?= htmlspecialchars($status) ?>">
<td>
<div class="item-name"><?= htmlspecialchars($name) ?></div>
<?php if ($label): ?><div class="item-sublabel"><?= htmlspecialchars($label) ?></div><?php endif; ?>
<?php if ($type === 'locator' && $method && $method !== 'xpath'): ?>
<div class="item-method"><?= htmlspecialchars($method) ?></div>
<?php endif; ?>
</td>
<td>
<span class="badge badge-<?= $type ?>"><?= ucfirst($type) ?></span>
</td>
<td class="val-cell">
<?php if ($display_value !== null): ?>
<?= htmlspecialchars((string)$display_value) ?>
<?php elseif ($status === 'error'): ?>
<span class="val-error" title="<?= htmlspecialchars($errMsg) ?>">ERROR</span>
<?php else: ?>
<span class="val-null">—</span>
<?php endif; ?>
</td>
<td class="spark-cell">
<?= render_sparkline_fixed($history) ?>
</td>
<td>
<span class="status-pill status-<?= $status ?>">
<span class="dot dot-<?= $status ?>"></span>
<?= strtoupper($status) ?>
</span>
</td>
<td class="time-cell"><?= $updated ? time_ago($updated) : '—' ?></td>
<td>
<span class="sc-chip" onclick="copyShortcode('<?= htmlspecialchars($name) ?>')" title="Copy shortcode">
<span class="ms">content_copy</span>[<?= htmlspecialchars($name) ?>]
</span>
</td>
<td class="actions">
<button class="btn-icon" onclick="doRefresh('<?= htmlspecialchars($name) ?>')" title="Refresh">
<span class="ms">refresh</span>
</button>
<button class="btn-icon" onclick="openEdit('<?= htmlspecialchars($name) ?>')" title="Edit">
<span class="ms">edit</span>
</button>
<button class="btn-icon danger" onclick="doDelete('<?= htmlspecialchars($name) ?>')" title="Delete">
<span class="ms">delete</span>
</button>
</td>
</tr>
<?php endforeach; endif; ?>
</tbody>
</table>
</div>
</div>
</div>
</div>
<!-- MODAL -->
<div class="modal-overlay" id="itemModal">
<div class="modal-panel">
<div class="modal-hdr">
<span class="ms" id="modalIcon" style="color:var(--accent)">add_circle</span>
<span class="modal-title" id="modalTitle">Add Item</span>
<button class="btn-icon" onclick="closeModal()"><span class="ms">close</span></button>
</div>
<div class="modal-body" id="modalBody"></div>
<div class="modal-ftr">
<button class="btn" onclick="closeModal()">Cancel</button>
<button class="btn btn-primary" id="btnSave" onclick="doSave()">
<span class="ms">save</span>Save
</button>
</div>
</div>
</div>
<div class="toast-wrap" id="toastWrap"></div>
<!-- ═══════════════════ JAVASCRIPT ═══════════════════ -->
<script>
'use strict';
const ALL_ITEMS = <?= $all_items_json ?>;
let editingName = null;
let filterType = '';
let filterStatus = '';
let searchQ = '';
let pipelineIdx = 0;
let currentMethod = 'xpath';
const toggleSidebar = () => {
document.getElementById('sidebar').classList.toggle('open');
document.getElementById('sbOverlay').classList.toggle('open');
};
const closeSidebar = () => {
document.getElementById('sidebar').classList.remove('open');
document.getElementById('sbOverlay').classList.remove('open');
};
function navFilter(val, kind) {
filterType = kind === 'type' ? val : '';
filterStatus = kind === 'status' ? val : '';
document.querySelectorAll('.nav-item').forEach(el => el.classList.remove('active'));
document.getElementById('nav-' + val)?.classList.add('active');
applyFilters();
if (window.innerWidth <= 768) closeSidebar();
}
function doSearch(q) {
searchQ = q.toLowerCase();
applyFilters();
}
function applyFilters() {
document.querySelectorAll('.irow').forEach(row => {
const t = row.dataset.type;
const s = row.dataset.status;
const n = row.dataset.name.toLowerCase();
const sub = (row.querySelector('.item-sublabel')?.textContent || '').toLowerCase();
let ok = true;
if (filterType && t !== filterType) ok = false;
if (filterStatus && s !== filterStatus) ok = false;
if (searchQ && !n.includes(searchQ) && !sub.includes(searchQ)) ok = false;
row.style.display = ok ? '' : 'none';
});
}
function copyShortcode(name) {
const sc = '[' + name + ']';
navigator.clipboard?.writeText(sc).then(() => toast('Copied ' + sc, 'ok', 'content_copy'))
.catch(() => {
const el = Object.assign(document.createElement('textarea'), {value: sc});
document.body.appendChild(el); el.select(); document.execCommand('copy'); el.remove();
toast('Copied ' + sc, 'ok', 'content_copy');
});
}
async function doRefresh(name) {
toast('Refreshing ' + name + '…', '', 'refresh');
const r = await apiFetch('?action=refresh&name=' + encodeURIComponent(name));
if (r?.success) { toast(name + ' refreshed!', 'ok', 'check_circle'); setTimeout(() => location.reload(), 500); }
else toast('Error: ' + (r?.error || 'failed'), 'err', 'error');
}
async function doRefreshAll() {
const btn = document.getElementById('btnRefreshAll');
btn.innerHTML = '<span class="spin"></span> Refreshing…';
btn.disabled = true;
await apiFetch('?action=refresh_all');
toast('All items refreshed', 'ok', 'check_circle');
setTimeout(() => location.reload(), 600);
}
function doDelete(name) {
if (!confirm('Delete "' + name + '"? This cannot be undone.')) return;
apiFetch('?action=delete&name=' + encodeURIComponent(name))
.then(r => {
if (r?.success) {
toast(name + ' deleted', '', 'delete');
document.querySelector('.irow[data-name="' + name + '"]')?.remove();
} else toast('Error: ' + (r?.error || ''), 'err', 'error');
});
}
function openAdd() {
editingName = null;
document.getElementById('modalIcon').textContent = 'add_circle';
document.getElementById('modalTitle').textContent = 'Add Item';
buildForm(null, null);
document.getElementById('itemModal').classList.add('open');
}
function openEdit(name) {
editingName = name;
document.getElementById('modalIcon').textContent = 'edit';
document.getElementById('modalTitle').textContent = 'Edit — ' + name;
buildForm(name, ALL_ITEMS[name]);
document.getElementById('itemModal').classList.add('open');
}
function closeModal() {
document.getElementById('itemModal').classList.remove('open');
editingName = null;
}
function buildForm(name, item) {
const type = item?.type || 'locator';
currentMethod = item?.method || 'xpath';
document.getElementById('modalBody').innerHTML = `
<div class="fg">
<label class="flabel">Item Name <span class="req">*</span></label>
<input class="finput mono" id="f_name" placeholder="e.g. usd_npr" value="${x(name||'')}" ${name ? 'readonly style="opacity:.6;cursor:not-allowed"' : ''}>
<div class="fhint">Letters, numbers and underscores only. Used as shortcode key.</div>
</div>
<div class="fg">
<label class="flabel">Label <span style="color:var(--c-muted)">(optional)</span></label>
<input class="finput" id="f_label" placeholder="Human-readable label" value="${x(item?.label||'')}">
</div>
<div class="fg">
<label class="flabel">Type <span class="req">*</span></label>
<select class="fselect" id="f_type" onchange="buildTypeFields()">
${['locator','calculator','constant','formatter','switch'].map(t =>
`<option value="${t}" ${type===t?'selected':''}>${t.charAt(0).toUpperCase()+t.slice(1)} — ${typeDesc(t)}</option>`
).join('')}
</select>
</div>
<div id="typeFields"></div>
`;
buildTypeFields(item);
}
function typeDesc(t) {
return {locator:'Scrape value from URL',calculator:'Arithmetic formula',constant:'Static number',formatter:'Format pipeline',switch:'Conditional value'}[t]||'';
}
function setMethod(m) {
currentMethod = m;
document.querySelectorAll('.method-tab').forEach(t => t.classList.toggle('active', t.dataset.method === m));
buildMethodFields();
}
function buildTypeFields(item) {
const type = document.getElementById('f_type').value;
if (!item && editingName) item = ALL_ITEMS[editingName];
let h = '';
if (type === 'locator') {
currentMethod = item?.method || 'xpath';
h = `
<div class="form-section">Extraction Method</div>
<div class="method-tabs">
<div class="method-tab ${currentMethod==='xpath'?'active':''}" data-method="xpath" onclick="setMethod('xpath')">
<span class="ms">code</span>XPath
<small>HTML DOM selector</small>
</div>
<div class="method-tab ${currentMethod==='regex'?'active':''}" data-method="regex" onclick="setMethod('regex')">
<span class="ms">manage_search</span>Regex
<small>Pattern in source</small>
</div>
<div class="method-tab ${currentMethod==='json_path'?'active':''}" data-method="json_path" onclick="setMethod('json_path')">
<span class="ms">data_object</span>JSON Path
<small>API endpoint</small>
</div>
</div>
<div class="fg">
<label class="flabel">URL <span class="req">*</span></label>
<input class="finput" id="f_url" type="url" placeholder="https://example.com/rates" value="${x(item?.url||'')}">
</div>
<div id="methodFields"></div>
`;
document.getElementById('typeFields').innerHTML = h;
buildMethodFields(item);
return;
} else if (type === 'calculator') {
const deps = extractDeps(item?.formula||'');
h = `
<div class="form-section">Formula</div>
<div class="fg">
<label class="flabel">Expression <span class="req">*</span></label>
<input class="finput mono" id="f_formula" placeholder="[usd_npr] * [gold_oz]" value="${x(item?.formula||'')}" oninput="updateDeps()">
<div class="fhint">Reference items with [item_name]. Supports + − × ÷ and parentheses.</div>
</div>
<div id="depsBox">${renderDeps(deps)}</div>
`;
} else if (type === 'constant') {
h = `
<div class="form-section">Value</div>
<div class="fg">
<label class="flabel">Static Value <span class="req">*</span></label>
<input class="finput mono" id="f_value" type="number" step="any" placeholder="133.45" value="${x(item?.value!==undefined?item.value:'')}">
<div class="fhint">Stored as-is. Edit any time from the dashboard.</div>
</div>
`;
} else if (type === 'formatter') {
pipelineIdx = 0;
h = `
<div class="form-section">Source & Pipeline</div>
<div class="fg">
<label class="flabel">Source Item <span class="req">*</span></label>
<select class="fselect" id="f_source">
<option value="">— Select source item —</option>
${Object.keys(ALL_ITEMS).map(k => `<option value="${x(k)}" ${item?.source===k?'selected':''}>${x(k)}</option>`).join('')}
</select>
</div>
<div class="fg">
<label class="flabel">Pipeline Steps</label>
<div id="pipelineWrap">${(item?.pipeline||[]).map((s,i)=>renderPipeStep(s,i)).join('')}</div>
<button class="btn btn-sm mt8" onclick="addPipeStep()"><span class="ms">add</span>Add Step</button>
<div class="fhint">Steps run in order. Prefix/Suffix produce string output.</div>
</div>
`;
} else if (type === 'switch') {
h = `
<div class="form-section">Condition</div>
<div class="fg">
<label class="flabel">Condition Expression <span class="req">*</span></label>
<input class="finput mono" id="f_condition" placeholder="[usd_npr] > 133" value="${x(item?.condition||'')}">
<div class="fhint">Use [item_name]. Operators: > < >= <= == !=</div>
</div>
<div class="fg-row">
<div class="fg">
<label class="flabel">If TRUE → Item</label>
<select class="fselect" id="f_if_true">
<option value="">— Select —</option>
${Object.keys(ALL_ITEMS).map(k => `<option value="${x(k)}" ${item?.if_true===k?'selected':''}>${x(k)}</option>`).join('')}
</select>
</div>
<div class="fg">
<label class="flabel">If FALSE → Item</label>
<select class="fselect" id="f_if_false">
<option value="">— Select —</option>
${Object.keys(ALL_ITEMS).map(k => `<option value="${x(k)}" ${item?.if_false===k?'selected':''}>${x(k)}</option>`).join('')}
</select>
</div>
</div>
`;
}
document.getElementById('typeFields').innerHTML = h;
}
/* ── BUILD METHOD-SPECIFIC FIELDS FOR LOCATOR ── */
function buildMethodFields(item) {
if (!item && editingName) item = ALL_ITEMS[editingName];
const el = document.getElementById('methodFields');
if (!el) return;
let h = '';
if (currentMethod === 'xpath') {
h = `
<div class="fg">
<label class="flabel">XPath — Primary <span class="req">*</span></label>
<input class="finput mono" id="f_xpath" placeholder="//span[@class='price']" value="${x(item?.xpath||'')}">
</div>
<div class="fg">
<label class="flabel">XPath — Fallback 2</label>
<input class="finput mono" id="f_xpath2" placeholder="//div[@id='rate']" value="${x(item?.xpath2||'')}">
<div class="fhint">Used if primary fails (resilient to DOM changes)</div>
</div>
<div class="fg">
<label class="flabel">XPath — Fallback 3</label>
<input class="finput mono" id="f_xpath3" placeholder="" value="${x(item?.xpath3||'')}">
</div>
${testerHTML('xpath')}
`;
} else if (currentMethod === 'regex') {
h = `
<div class="info-box">
<strong>When to use Regex:</strong> For JS-rendered sites where XPath finds empty nodes.
The data is often embedded in a <code><script></code> tag as JSON or a JS variable.<br><br>
<strong>How to find the pattern:</strong><br>
1. Open the site → View Page Source (Ctrl+U)<br>
2. Search (Ctrl+F) for the value you want<br>
3. Write a regex with one <code>(capture group)</code> around the number<br><br>
<strong>Examples:</strong><br>
<code>/"buying"\s*:\s*"?([\d,.]+)/</code><br>
<code>/var\s+rate\s*=\s*"?([\d,.]+)/</code>
</div>
<div class="fg">
<label class="flabel">Regex Pattern — Primary <span class="req">*</span></label>
<input class="finput mono" id="f_regex" placeholder='/"buying"\\s*:\\s*"?([\\d,.]+)/' value="${x(item?.regex||'')}">
<div class="fhint">Must have one capture group <code>(...)</code> around the number. Use PCRE syntax with delimiters.</div>
</div>
<div class="fg">
<label class="flabel">Regex — Fallback 2</label>
<input class="finput mono" id="f_regex2" placeholder="" value="${x(item?.regex2||'')}">
</div>
<div class="fg">
<label class="flabel">Regex — Fallback 3</label>
<input class="finput mono" id="f_regex3" placeholder="" value="${x(item?.regex3||'')}">
</div>
${testerHTML('regex')}
`;
} else if (currentMethod === 'json_path') {
h = `
<div class="info-box">
<strong>When to use JSON Path:</strong> When the site's JS calls an API that returns JSON.
Hit the API URL directly instead of scraping HTML.<br><br>
<strong>How to find the API:</strong><br>
1. Open DevTools → Network tab → filter by "XHR" or "Fetch"<br>
2. Reload the page and look for requests returning exchange rate data<br>
3. Copy the API URL and use dot notation to traverse the JSON<br><br>
<strong>Example:</strong> URL: <code>https://example.com/api/rates</code><br>
Path: <code>data.payload.rates.0.buy</code>
</div>
<div class="fg">
<label class="flabel">JSON Path — Primary <span class="req">*</span></label>
<input class="finput mono" id="f_json_path" placeholder="data.payload.rates.0.buy" value="${x(item?.json_path||'')}">
<div class="fhint">Dot-separated path. Use numbers for array indices: <code>data.0.rates.usd</code></div>
</div>
<div class="fg">
<label class="flabel">JSON Path — Fallback</label>
<input class="finput mono" id="f_json_path2" placeholder="" value="${x(item?.json_path2||'')}">
</div>
${testerHTML('json_path')}
`;
}
el.innerHTML = h;
}
/* ── TESTER HTML (shared across methods) ── */
function testerHTML(method) {
const labels = {xpath:'XPath',regex:'Regex',json_path:'JSON Path'};
return `
<div class="tester-wrap">
<div class="tester-hdr" onclick="toggleTester(this)">
<span class="ms">science</span>
Live ${labels[method]} Tester
<span style="margin-left:auto;font-size:11px;color:var(--c-sub)" id="testerToggleLbl">▼ Expand</span>
</div>
<div class="tester-body" id="testerBody" style="display:none">
<div class="fhint" style="margin-bottom:8px">Uses URL and primary expression from above. Nothing is saved.</div>
<button class="btn btn-sm" id="btnTest" onclick="runExtractTest()">
<span class="ms">play_arrow</span>Run Test
</button>
<div id="testerResult"></div>
</div>
</div>
`;
}
function toggleTester(hdr) {
const body = document.getElementById('testerBody');
const lbl = document.getElementById('testerToggleLbl');
const open = body.style.display !== 'none';
body.style.display = open ? 'none' : 'block';
if (lbl) lbl.textContent = open ? '▼ Expand' : '▲ Collapse';
}
/* ── UNIVERSAL TESTER ── */
async function runExtractTest() {
const url = document.getElementById('f_url')?.value?.trim();
let expr = '';
if (currentMethod === 'xpath') expr = document.getElementById('f_xpath')?.value?.trim();
if (currentMethod === 'regex') expr = document.getElementById('f_regex')?.value?.trim();
if (currentMethod === 'json_path') expr = document.getElementById('f_json_path')?.value?.trim();
const res = document.getElementById('testerResult');
const btn = document.getElementById('btnTest');
if (!url || !expr) {
res.innerHTML = '<div class="tester-result tester-err">Fill in URL and the primary expression first.</div>';
return;
}
btn.innerHTML = '<span class="spin"></span> Testing…';
btn.disabled = true;
res.innerHTML = '';
const data = await apiFetch('?action=test_extract', {
method: 'POST',
body: JSON.stringify({url, method: currentMethod, expr}),
headers: {'Content-Type':'application/json'}
});
btn.innerHTML = '<span class="ms">play_arrow</span>Run Test';
btn.disabled = false;
if (data?.success) {
res.innerHTML = `<div class="tester-result tester-ok">✓ Raw text: "${data.raw}"\n✓ Parsed value: ${data.value}</div>`;
} else {
let errHtml = `<div class="tester-result tester-err">✗ ${data?.error || 'Unknown error'}`;
if (data?.raw) errHtml += `\n\nRaw match: "${data.raw}"`;
errHtml += `</div>`;
res.innerHTML = errHtml;
}
}
/* ── DEPS ── */
function extractDeps(formula) {
return [...new Set((formula.match(/\[([a-zA-Z0-9_]+)\]/g)||[]).map(s=>s.slice(1,-1)))];
}
function renderDeps(deps) {
if (!deps.length) return '';
return `<div style="font-size:11px;color:var(--c-sub);margin-bottom:5px">Dependencies:</div>
<div class="deps-wrap">${deps.map(d=>`<span class="dep-chip">[${x(d)}]</span>`).join('')}</div>`;
}
function updateDeps() {
const f = document.getElementById('f_formula')?.value||'';
const el = document.getElementById('depsBox');
if (el) el.innerHTML = renderDeps(extractDeps(f));
}
/* ── PIPELINE ── */
const PIPE_OPS = ['round','floor','ceil','abs','multiply','divide','prefix','suffix','number_format'];
function renderPipeStep(val, idx) {
const [op, arg] = val.split(':', 2);
pipelineIdx = Math.max(pipelineIdx, idx + 1);
return `<div class="pipeline-step" id="ps${idx}">
<select class="fselect" id="psop${idx}">
${PIPE_OPS.map(o=>`<option value="${o}" ${op===o?'selected':''}>${o}</option>`).join('')}
</select>
<input class="finput mono" id="psarg${idx}" placeholder="arg" value="${x(arg||'')}">
<button class="btn-icon danger" onclick="removePipeStep(${idx})"><span class="ms">remove_circle</span></button>
</div>`;
}
function addPipeStep() {
const wrap = document.getElementById('pipelineWrap');
const idx = pipelineIdx++;
const d = document.createElement('div');
d.innerHTML = renderPipeStep('round:', idx);
wrap.appendChild(d.firstElementChild);
}
function removePipeStep(idx) { document.getElementById('ps'+idx)?.remove(); }
function collectPipeline() {
const steps = [];
document.querySelectorAll('#pipelineWrap .pipeline-step').forEach(row => {
const id = row.id.replace('ps','');
const op = document.getElementById('psop'+id)?.value;
const arg = document.getElementById('psarg'+id)?.value||'';
if (op) steps.push(op + (arg ? ':'+arg : ''));
});
return steps;
}
/* ── SAVE ── */
async function doSave() {
const type = document.getElementById('f_type')?.value;
const name = document.getElementById('f_name')?.value?.trim();
const label = document.getElementById('f_label')?.value?.trim();
if (!name) return toast('Name is required', 'err', 'error');
if (!/^[a-zA-Z0-9_]+$/.test(name)) return toast('Name: letters, numbers, underscores only', 'err', 'error');
const payload = {name, type, label};
switch (type) {
case 'locator':
payload.url = document.getElementById('f_url')?.value?.trim();
payload.method = currentMethod;
if (!payload.url) return toast('URL is required', 'err', 'error');
if (currentMethod === 'xpath') {
payload.xpath = document.getElementById('f_xpath')?.value?.trim();
payload.xpath2 = document.getElementById('f_xpath2')?.value?.trim();
payload.xpath3 = document.getElementById('f_xpath3')?.value?.trim();
if (!payload.xpath) return toast('XPath expression is required', 'err', 'error');
} else if (currentMethod === 'regex') {
payload.regex = document.getElementById('f_regex')?.value?.trim();
payload.regex2 = document.getElementById('f_regex2')?.value?.trim();
payload.regex3 = document.getElementById('f_regex3')?.value?.trim();
if (!payload.regex) return toast('Regex pattern is required', 'err', 'error');
} else if (currentMethod === 'json_path') {
payload.json_path = document.getElementById('f_json_path')?.value?.trim();
payload.json_path2 = document.getElementById('f_json_path2')?.value?.trim();
if (!payload.json_path) return toast('JSON path is required', 'err', 'error');
}
break;
case 'calculator':
payload.formula = document.getElementById('f_formula')?.value?.trim();
if (!payload.formula) return toast('Formula is required', 'err', 'error');
break;
case 'constant':
payload.value = document.getElementById('f_value')?.value;
if (payload.value === '' || payload.value === undefined) return toast('Value is required', 'err', 'error');
break;
case 'formatter':
payload.source = document.getElementById('f_source')?.value;
payload.pipeline = collectPipeline();
if (!payload.source) return toast('Source item is required', 'err', 'error');
break;
case 'switch':
payload.condition = document.getElementById('f_condition')?.value?.trim();
payload.if_true = document.getElementById('f_if_true')?.value;
payload.if_false = document.getElementById('f_if_false')?.value;
if (!payload.condition || !payload.if_true || !payload.if_false)
return toast('All switch fields are required', 'err', 'error');
break;
}
const btn = document.getElementById('btnSave');
btn.innerHTML = '<span class="spin"></span> Saving…'; btn.disabled = true;
const r = await apiFetch('?action=save', {
method: 'POST',
body: JSON.stringify(payload),
headers: {'Content-Type':'application/json'}
});
btn.innerHTML = '<span class="ms">save</span>Save'; btn.disabled = false;
if (r?.success) {
toast('Item saved!', 'ok', 'check_circle');
closeModal();
setTimeout(() => location.reload(), 500);
} else {
toast(r?.error || 'Save failed', 'err', 'error');
}
}
async function apiFetch(url, opts={}) {
try {
const r = await fetch(url, opts);
return await r.json();
} catch(e) {
toast('Network error: ' + e.message, 'err', 'wifi_off');
return null;
}
}
function toast(msg, kind, icon) {
const w = document.getElementById('toastWrap');
const t = document.createElement('div');
t.className = 'toast' + (kind ? ' '+kind : '');
t.innerHTML = `<span class="ms">${icon||'info'}</span>${msg}`;
w.appendChild(t);
setTimeout(() => t.style.opacity = '0', 3200);
setTimeout(() => t.remove(), 3500);
}
function x(s) {
return String(s)
.replace(/&/g,'&').replace(/"/g,'"')
.replace(/</g,'<').replace(/>/g,'>');
}
document.getElementById('itemModal').addEventListener('click', e => {
if (e.target === e.currentTarget) closeModal();
});
document.addEventListener('keydown', e => { if (e.key === 'Escape') closeModal(); });
</script>
</body>
</html>
<?php
} // end render_dashboard
themefile function.php in wordress
<?php
/**
* ╔════════════════════════════════════════════════════╗
* ║ DataEngine — WordPress Integration Snippet ║
* ╚════════════════════════════════════════════════════╝
*/
// ── CONFIGURATION ──────────────────────────────────────────
if (!defined('DATAENGINE_URL')) {
define('DATAENGINE_URL', 'https://sushilparajuli.com/apps/fetcher/engine.php');
}
if (!defined('DATAENGINE_WP_CACHE_SECS')) {
define('DATAENGINE_WP_CACHE_SECS', 3600);
}
// ── SHORTCODE HANDLER ──────────────────────────────────────
add_shortcode('data', 'dataengine_shortcode');
function dataengine_shortcode($atts, $content = null)
{
$atts = shortcode_atts([
'name' => '',
'format' => 'formatted',
'decimals' => '2',
'fallback' => 'N/A',
], $atts, 'data');
$name = sanitize_key($atts['name']);
if (empty($name)) {
return esc_html($atts['fallback']);
}
$cache_key = 'de_' . $name . '_' . $atts['format'] . '_' . $atts['decimals'];
if (DATAENGINE_WP_CACHE_SECS > 0) {
$cached = get_transient($cache_key);
if ($cached !== false) {
return $cached;
}
}
$url = add_query_arg([
'action' => 'get',
'name' => $name,
'format' => sanitize_text_field($atts['format']),
'decimals' => absint($atts['decimals']),
'fallback' => urlencode($atts['fallback']),
], DATAENGINE_URL);
$response = wp_remote_get($url, [
'timeout' => 10,
'user-agent' => 'WordPress DataEngine Client/1.0',
'sslverify' => false,
]);
if (is_wp_error($response)) {
return esc_html($atts['fallback']);
}
$code = wp_remote_retrieve_response_code($response);
if ($code !== 200) {
return esc_html($atts['fallback']);
}
$value = trim(wp_remote_retrieve_body($response));
if (empty($value)) {
return esc_html($atts['fallback']);
}
if (DATAENGINE_WP_CACHE_SECS > 0) {
set_transient($cache_key, $value, DATAENGINE_WP_CACHE_SECS);
}
return $value;
}
// ── CACHE PURGE HELPER ─────────────────────────────────────
function dataengine_flush_cache()
{
global $wpdb;
$wpdb->query(
"DELETE FROM {$wpdb->options}
WHERE option_name LIKE '_transient_de_%'
OR option_name LIKE '_transient_timeout_de_%'"
);
}
// ── ELEMENTOR: process shortcodes in content ───────────────
add_filter('elementor/frontend/the_content', 'do_shortcode');
// ── AUTO-REGISTER EACH ITEM AS ITS OWN SHORTCODE ──────────
// Allows [npr_billion] instead of [data name="npr_billion"]
add_action('init', function() {
$data_file = ABSPATH . 'apps/fetcher/data.json';
if (!file_exists($data_file)) return;
$data = json_decode(file_get_contents($data_file), true);
if (empty($data['items'])) return;
foreach (array_keys($data['items']) as $item_name) {
add_shortcode($item_name, function($atts) use ($item_name) {
$atts = shortcode_atts([
'format' => 'formatted',
'decimals' => '2',
'fallback' => 'N/A',
], $atts);
return dataengine_shortcode(array_merge(
['name' => $item_name],
$atts
));
});
}
});
// ── ELEMENTOR DYNAMIC TAG (optional) ──────────────────────
add_action('elementor/dynamic_tags/register', function($dynamic_tags) {
if (!class_exists('\\Elementor\\Core\\DynamicTags\\Tag')) return;
class DataEngine_Dynamic_Tag extends \Elementor\Core\DynamicTags\Tag
{
public function get_name() { return 'dataengine-value'; }
public function get_title() { return 'DataEngine Value'; }
public function get_group() { return 'site'; }
public function get_categories() { return [\Elementor\Modules\DynamicTags\Module::TEXT_CATEGORY]; }
protected function register_controls()
{
$this->add_control('item_name', [
'label' => 'Item Name',
'type' => \Elementor\Controls_Manager::TEXT,
'default' => '',
'placeholder' => 'e.g. usd_npr',
]);
$this->add_control('format', [
'label' => 'Format',
'type' => \Elementor\Controls_Manager::SELECT,
'default' => 'formatted',
'options' => [
'formatted' => 'Formatted (e.g. 133.45)',
'raw' => 'Raw number',
'integer' => 'Integer (no decimals)',
],
]);
$this->add_control('fallback', [
'label' => 'Fallback',
'type' => \Elementor\Controls_Manager::TEXT,
'default' => 'N/A',
]);
}
public function render()
{
$settings = $this->get_settings();
echo dataengine_shortcode([
'name' => $settings['item_name'] ?? '',
'format' => $settings['format'] ?? 'formatted',
'fallback' => $settings['fallback'] ?? 'N/A',
]);
}
}
$dynamic_tags->register(new DataEngine_Dynamic_Tag());
});