Oda/managing-apps
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Postgres (#30)

Browse Source 
* Add postgres

* Migrate users

* Migrate geneticRequest

* Try to fix Concurrent call

* Fix asyncawait

* Fix async and concurrent

* Migrate backtests

* Add cache for user by address

* Fix backtest migration

* Fix not open connection

* Fix backtest command error

* Fix concurrent

* Fix all concurrency

* Migrate TradingRepo

* Fix scenarios

* Migrate statistic repo

* Save botbackup

* Add settings et moneymanagement

* Add bot postgres

* fix a bit more backups

* Fix bot model

* Fix loading backup

* Remove cache market for read positions

* Add workers to postgre

* Fix workers api

* Reduce get Accounts for workers

* Migrate synth to postgre

* Fix backtest saved

* Remove mongodb

* botservice decorrelation

* Fix tradingbot scope call

* fix tradingbot

* fix concurrent

* Fix scope for genetics

* Fix account over requesting

* Fix bundle backtest worker

* fix a lot of things

* fix tab backtest

* Remove optimized moneymanagement

* Add light signal to not use User and too much property

* Make money management lighter

* insert indicators to awaitable

* Migrate add strategies to await

* Refactor scenario and indicator retrieval to use asynchronous methods throughout the application

* add more async await

* Add services

* Fix and clean

* Fix bot a bit

* Fix bot and add message for cooldown

* Remove fees

* Add script to deploy db

* Update dfeeploy script

* fix script

* Add idempotent script and backup

* finish script migration

* Fix did user and agent name on start bot
This commit is contained in:
Oda
2025-07-27 15:42:17 +02:00
committed by GitHub
parent 361bfbf6e8
commit 422fecea7b
294 changed files with 23953 additions and 7272 deletions
Download Patch File Download Diff File Expand all files Collapse all files

222
scripts/README.md
View File

@@ -1,191 +1,63 @@
# Scripts
# 🚀 Safe Database Migration Script
This directory contains utility scripts for the project.
A clean, focused script for safe database migrations with automatic backup and cleanup.
## Add JS Extensions Script
## 📋 Features
The `add-js-extensions.mjs` script adds `.js` extensions to import statements in JavaScript and TypeScript files, and also adds the required JSON import assertions.
- ✅ **Connectivity Check**: Verifies database connection before proceeding
- ✅ **Automatic Backup**: Creates backup with retry logic (3 attempts)
- ✅ **Safety First**: Exits if backup fails - no migration without backup
- ✅ **Smart PostgreSQL**: Uses Docker PostgreSQL client if `psql` not available
- ✅ **Migration**: Runs Entity Framework migrations
- ✅ **Verification**: Checks migration success
- ✅ **Cleanup**: Keeps only last 5 backups per environment
### Why This Script?
When working with ES Modules in Node.js:
1. Imports of local files require explicit file extensions
2. JSON imports require an `assert { type: 'json' }` assertion
This script automates both processes to ensure your code is ESM-compatible.
### Usage
Run the script with npm:
## 🛠️ Usage
```bash
# Fix imports in the src directory (default)
npm run fix-imports
# Development environment
./safe-migrate.sh Development
# Fix imports in a specific directory
npm run fix-imports-dir -- path/to/directory
# Sandbox environment
./safe-migrate.sh Sandbox
# Production environment
./safe-migrate.sh Production
```
Or run the script directly:
## 🔄 Process Flow
```bash
# Fix imports in the src directory (default)
node scripts/add-js-extensions.mjs
1. **Environment Validation**: Validates environment parameter
2. **Connectivity Check**: Tests database connection
3. **Backup Creation**: Creates backup with retry logic
4. **Migration**: Runs pending migrations
5. **Verification**: Checks migration success
6. **Cleanup**: Removes old backups (keeps last 5)
# Fix imports in a specific directory
node scripts/add-js-extensions.mjs path/to/directory
## 🚨 Safety Features
- **Mandatory Backup**: Script exits if backup cannot be created
- **Retry Logic**: 3 backup attempts with 5-second delays
- **Error Handling**: Clear error messages and exit codes
- **Logging**: Detailed logs for troubleshooting
## 📊 Output
```
✅ Database connectivity test passed
✅ Database backup created: ./backups/managing_Development_backup_20250726_043047.sql
✅ Database migration completed successfully
✅ Database schema verification passed
✅ Kept last 5 backups for Development environment
```
### What This Script Does
## 🔧 Prerequisites
1. Recursively scans all JavaScript and TypeScript files in the specified directory
2. Identifies import statements with relative paths (starting with `./` or `../`) that don't have extensions and adds `.js` extensions
3. Identifies JSON imports that are missing the required assertion and adds `assert { type: 'json' }`
4. Provides a summary of files modified and any errors encountered
- Docker (for PostgreSQL client)
- .NET 8.0 SDK
- Access to target database
### Examples
## 📁 Files
Before:
```javascript
import { bigMath } from "./bigmath";
import data from "./data.json";
```
After:
```javascript
import { bigMath } from "./bigmath.js";
import data from "./data.json" assert { type: 'json' };
```
### Important Notes
- The script only modifies imports with relative paths (starting with `./` or `../`)
- It skips imports that already have a file extension (except for JSON files)
- It adds `.js` extensions to extensionless imports
- It adds `assert { type: 'json' }` to JSON imports that don't already have it
- It handles regular imports, dynamic imports, and exports
## Remove JSON Assertions Script
The `remove-json-assertions.mjs` script removes `assert { type: 'json' }` assertions from JSON imports.
### Why This Script?
Different JavaScript environments have different requirements for JSON imports:
1. Some newer environments require the `assert { type: 'json' }` assertion
2. Others don't support or need these assertions
This script removes these assertions to improve compatibility with environments that don't need them.
### Usage
Run the script with npm:
```bash
# Remove JSON assertions
npm run remove-json-assertions
# Run both import fixes and assertion removal in one command
npm run prepare-code
```
Or run the script directly:
```bash
node scripts/remove-json-assertions.mjs
```
### What This Script Does
1. Recursively scans all JavaScript and TypeScript files in the project
2. Identifies JSON imports with `assert { type: 'json' }` assertions
3. Removes the assertions while preserving the import statements
4. Provides a summary of files modified
### Examples
Before:
```javascript
import data from "./data.json" assert { type: 'json' };
```
After:
```javascript
import data from "./data.json";
```
### Important Notes
- The script only modifies JSON imports with assertions
- It preserves all other import statements
- It works in conjunction with the add-js-extensions script
- These scripts can be run in sequence to first fix imports then remove assertions
## Update JSON Imports Script
The `update-json-imports.mjs` script updates JSON imports to use the modern `with { type: "json" }` syntax.
### Why This Script?
Different JavaScript environments have different requirements for JSON imports:
1. Older environments used `assert { type: 'json' }` for JSON imports
2. Modern JavaScript environments now use the `with { type: "json" }` syntax
This script updates your codebase to use the newer, more standardized approach.
### Usage
Run the script with npm:
```bash
# Update JSON import syntax
npm run update-json-imports
# Run both import fixing and JSON import updating in one command
npm run prepare-code
```
Or run the script directly:
```bash
node scripts/update-json-imports.mjs
```
### What This Script Does
1. Recursively scans all JavaScript and TypeScript files in the project
2. Identifies JSON imports using either:
- The older `assert { type: 'json' }` syntax
- No type assertion at all
- Erroneous dual syntax (`assert { type: 'json' } with { type: "json" }`)
3. Updates them to use the modern `with { type: "json" }` syntax
4. Provides a summary of files modified
### Examples
Before (old assert syntax):
```javascript
import data from "./data.json" assert { type: 'json' };
```
Before (no type assertion):
```javascript
import data from "./data.json";
```
Before (erroneous dual syntax):
```javascript
import data from "./data.json" assert { type: 'json' } with { type: "json" };
```
After (in all cases):
```javascript
import data from "./data.json" with { type: "json" };
```
### Important Notes
- The script updates all JSON imports to use the modern syntax
- It properly fixes cases where both old and new syntax are present
- It preserves all other import statements
- Files with properly formatted imports are not modified
- **Backups**: `./backups/managing_[Environment]_backup_[Timestamp].sql`
- **Logs**: `./migration_[Environment]_[Timestamp].log`