docker files fixes from liaqat

This commit is contained in:
alirehmani
2024-05-03 16:39:25 +05:00
commit 464a8730e8
587 changed files with 44288 additions and 0 deletions

75
assets/NSwagConfig.nswag Normal file
View File

@@ -0,0 +1,75 @@
{
"runtime": "Net60",
"defaultVariables": null,
"documentGenerator": {
"fromDocument": {
"url": "https://localhost:5001/swagger/v1/swagger.json",
"output": null,
"newLineBehavior": "Auto"
}
},
"codeGenerators": {
"openApiToTypeScriptClient": {
"className": "{controller}Client",
"moduleName": "",
"namespace": "",
"typeScriptVersion": 4.3,
"template": "Fetch",
"promiseType": "Promise",
"httpClass": "HttpClient",
"withCredentials": false,
"useSingletonProvider": false,
"injectionTokenType": "OpaqueToken",
"rxJsVersion": 6.0,
"dateTimeType": "Date",
"nullValue": "Undefined",
"generateClientClasses": true,
"generateClientInterfaces": false,
"generateOptionalParameters": false,
"exportTypes": true,
"wrapDtoExceptions": false,
"exceptionClass": "ApiException",
"clientBaseClass": null,
"wrapResponses": false,
"wrapResponseMethods": [],
"generateResponseClasses": true,
"responseClass": "SwaggerResponse",
"protectedMethods": [],
"configurationClass": null,
"useTransformOptionsMethod": false,
"useTransformResultMethod": false,
"generateDtoTypes": true,
"operationGenerationMode": "MultipleClientsFromFirstTagAndOperationId",
"markOptionalProperties": true,
"generateCloneMethod": false,
"typeStyle": "Interface",
"enumStyle": "Enum",
"useLeafType": false,
"classTypes": [],
"extendedClasses": [],
"extensionCode": null,
"generateDefaultValues": true,
"excludedTypeNames": [],
"excludedParameterNames": [],
"handleReferences": false,
"generateConstructorInterface": true,
"convertConstructorInterfaceData": false,
"importRequiredTypes": true,
"useGetBaseUrlMethod": false,
"baseUrlTokenName": "API_BASE_URL",
"queryNullValue": "",
"useAbortSignal": false,
"inlineNamedDictionaries": false,
"inlineNamedAny": false,
"includeHttpContext": false,
"templateDirectory": null,
"typeNameGeneratorType": null,
"propertyNameGeneratorType": null,
"enumNameGeneratorType": null,
"serviceHost": null,
"serviceSchemes": null,
"output": "Managing.WebApp/src/generated/",
"newLineBehavior": "Auto"
}
}
}

151
assets/Todo-v1.txt Normal file
View File

@@ -0,0 +1,151 @@
Principals features :
- Create X bots
- Each Trading bot can have multiple strategies
- Each Strategy is run for every new candle that the bot read from the database
- Use exchange worker to retrieve candles for a ticker
- Open one click a long with SL and 2 take profit with defined risk
V1 :
Part 1 : Running bot
- OK - Create no relationnal database to store all tickers history by exchanges
- OK - Feed the database with a Bot
- OK - On Bot motherclass, create a method accessible to child to retrieve the last candle since a prompted date
- OK - Create 3 Soldiers strategy
- OK - Create rule "CloseHigherThanThePreviousHigh"
- OK - Create rule "CloseLowerThanThePreviousLow"
Part 2 : Make a trade
- OK - Implement a GetBalance with the value in $
- OK - Calculate risk per for a given trade
- OK - Open a Long with defined risk level and exchange
- OK - Define and Open Stop loss with risk level
- OK - Define and open 2 Take Profit with risk level
- OK - Register generic candle for every exchanges in database
- OK - Create an endpoint to retrieve all order (with a filter class)
- OK - Create an endpoint to close an order for a given ExchangeOrderId
- OK - Implement OpenMarketTrade for Binance Futures
- OK - Implement GetPrice for Binance Futures
- OK - Implement GetTrade for Binance Futures
- OK - Binance : Handle TimeInForce for limit or market trade
Part 3 : Bot running RSI
- OK - Create RSI for 32period
- OK - Handle multiple divergence point for a candle range
- OK - Create rule to find a divergence for given RSI -> Confidence.High
- OK - Create an identifier for each bot based on name-datetime
- OK - Save price into Signal Object
Part 3.1
- OK - Each bot should be a manager keep in memory the trades executed and execute trade by calling openMarketPosition()
- OK - Create a mother class call TradeManager
- OK - This class should access to the signal return by the Scalping bot
- OK - This class should handle the trade management
- OK - Save Signals into database
Part 3.2 :
- OK - Create a unit test to backtest the bot execution
- OK - Get range candle list for a ticker
- OK - Each bot should have a backtest mode. This mode allow the caller to insert his own candles into the list. This mode will be used by the unit tests
- OK - Foreach candle, update the candle list of the bot and implicitly run all strategies
- OK - For each last signal for a ticker check if position is open. If not: Open position.
- OK - If an position is already open, update the position and check the price and trigger a close/flip if there is an inversed signal for that position.
- OK - If there is a inverted signal, close or flip the position depending of the confidence of the signal
- OK - Create a collection for registering the position with pnl, status
- OK - Save in trade table a reference to the position id, bot identifier
- OK - Create class to calculate PNL
- OK - openMarketPosition () should handle a parameter to make paper trading: just return the object that represent the trade executed by the exchange. That parameter will be set by bot in backtest mode
- OK - Add function to reopen trade that wasnt filled or whatever
- OK - Assert the fact that a list of positions have been created and ending in positive
- OK - Fix timestamp unix for getting klines for OneMinutes timeframe
- OK - Run bot on past candle
- OK - Extract trading logic from Bot class, to create a (TradingBot : Bot) class.
Part 4:
- Ok - Create RSIBearishDivergenceStrategy
- OK - Move Risk level config into appsettings to manipulate easily the user preference
- OK - Implement multi-timeframe to Define MoneyManagement for everyTimeFrame
- OK - Handle little quantity for trade by multiplying the quantity inside pnl object
- OK - Set the correct quantity for a trade who take profit #1
- OK - Add more tests for ProfitAndLoss object
- OK - Setup optionnal second take profit
- OK - Create DailyBot and Minutes bot to handle 5 ans 14 periods rsi
- OK - Run Bot for little TimeFrame
- OK - Implement GetTrade for an OrderId on Binance
- OK - Add discord connection
- OK - Expose all the API on discord bot
- OK - Run instance of bot for ADA, 5 periods, 15min
- OK - Refactoring bots unit tests to use same method than discord bot
- OK - Create CSV file for backtesting report
Part 5:
- OK - Improve open position from discord
- OK - Improve multi-risk levels handling
- OK - Improve divergence signal by handeling confidence
- OK - Implement ExchangeService.CloseTrade for exchanges
Part 6: webapp
- OK - Build front-end app with react
- OK - Build Tradingview component
- Update trading bot from signalr for everynew run => pass the BotHubContext to ITradingbot interface
- OK - Bots page to manage running bots
- - OK - Create a modal to start a bot
- - OK - On backtest page, display a button to start a bot based on defined backtest configuration
- - Handle stop and restart
- OK - Create scanner page to load all the ticker backtest base on a setup (timeframe, risk, day, bottype)
- OK - Save backtestResult into database to display it into Scanner page
- OK - Display backtest result on scanner page
- Scenario builder => A component to build nested trading strategy https://www.kindacode.com/article/react-typescript-drag-and-drop/
- - OK - Store scenario into database -> a scenario contain a list of defined strategy
- - OK - Dynamically insert scenario into ITradingBot
- - OK - Create a modal form to build a scenario that contains a list of strategies
- - OK - On backtest page, retrieve scenario list to launch a backtest based on it
Part 7: Improve trading
- OK - Handle fees
- OK - Create strategy to send rsi div signal on confirmation only
- OK - Make flipping position optionnal
- OK - Candle mapped on the wrong time => 18h30 should be 18h15
- KO - Enable backtesting on very long time range => No historical data more than 1500ohlc per timeframe on FTX
Part 8 : enhance front
- Settings page =>
- OK - Put money management into database to be able to modify it without rebuild/restart the api
- OK - Create a tab page on settings page to easily update monymanagement in db
- Improve card informations for bots
- - OK - Display the signals count for long and short
- - OK - Add button to toggle start and stop
- - OK - Add a toggle for isWatchingOnly
- OK - Add filter checkbox on Backtest/Moneymanagement to filter the list
- OK - Create a button to delete all backtests
- OK - Implement switch network debug/local
Part 9: Improve bot usage
- Save bot config to database to re-run bot on restart
- OK - Create endpoint to restart a bot
- Ok - Create endpoint to stop all bots
- OK - On signal, if a position is open => Same direction : expired signal | opposition direction : Flip open position
- OK - Implement a flipping function on an open position
- Ok - Track balance evolution to determine max drawdown
- OK - Determine best parameter for a given scenario : https://www.youtube.com/watch?v=jm-UfVPqUQo (Start with a unit test)
- OK - When a bot start up for a ticker, clean all limit order
- OK - When a bot start up for a ticker, clean open position
- OK - Before opening a trade clean limit order
- OK - Get status of position before closing it. The position might be already close by the exchange
Part 10: Improve performance
- OK - Calculate the strategies only on the last n*10 period
- OK - Merge divergence RSI strategy to calculate rsi only once by run
- KO - Prevent Order's notional must be no smaller than 5.0
- OK - Don't run update Signal and manage position if no new candle
Part 11: Extra
- OK - Create custom exception to handle error
- OK - Add ELK stacks
- OK - Display R/R used on backtestResult
- OK - Update Exchanges libraries
- OK - Integrate FTX exchange
- OK - Display wallet evolution
- OK - Remove API keys from git
- OK - Add seed for moneymanagement and scenario

199
assets/Todo-v2.md Normal file
View File

@@ -0,0 +1,199 @@
# V2
## Done
### Adaptive mode
- [x] Create a risk mode 'Adaptive' to be able to not use save moneymanagement settings
- [x] Adaptive mode will force Bot to determine SL by getting the previous highest price on the period (prevent choppy price on good entry)
- [x] Create account (type, exchange, key, secret)
- [x] Update trading bot to handle one account
- [x] Remove exchange from bot. Exchange will be set on the account
- [x] Update ftxClient to not use Managingbot subaccount
- [x] Update exchangeService to handle account
- [x] Dynamic Ticker selection base on best volumes on the exchanges
- [x] Create endpoint to get the ticker list for the frontend
- [x] Create method in application to get best bid and best ask
- [x] Submit only limit with IOC
- [x] Implement Poly to have a retry policy for limit order ioc execution => Unsure position is open
- [x] For openOrder inject a parameter to select the ioc mode. If risk is adaptive we just place the order a the price without forcing the position opening
- [x] Integrate STC signals
### Web3
- [x] Integrate wallet management from backend side
- [x] Integrate wagmi/connectkit
- [x] Sign message with connectkit
- [x] On wallet connect, check if token is not expired
- [x] User can have multiple address
- [x] User can create an account base on evm exchange
- [x] Hide key/secret if evm exchange is selected
- [x] Generate pub/priv key on evm account creation and store it into mongo
- [x] Get balance for an address
- [x] Handle multi-chain
- [x] Add arbitrum testnet for backend
- [x] User can withdraw found to the connected address of the user
- [x] Implement get balance with chainlink feed
- [x] Abstract subgraph management
- [x] Get price from multiple price feeds subgraph
- [x] Add to ExchangeService GetAvailableTicker()
- [x] Add get token list method to EvmManager
- [x] Setup worker to save candles into db
- [x] Open trade -> CreateIncreaseOrder
- [x] Close trade -> CreateDecreasePosition
- [x] Close all orders
- [x] Create SL & TP -> CreateDecreaseOrder
- [x] Get Fees -> Estimate Tx cost
- [x] Adapt GetBalance to retrieve stablecoin amount
- [x] Create worker to fetch GMX leaderboard -> LeaderboardWorker
- [x] Leaderboard will create PositionUpdate message bus to deposit a message with all information to open a trade
- [x] Create bot type CopyTrading to receive PositionUpdate bus message and open position based on criteria
- [x] CopyTrading bot will send an OpenPosition button on Discord if he's set to watchOnly
- [x] Fix : Trim address, give name to address, decrease amount bug,
- [x] Display best last backtests ran by backtester workers
- [x] Refactor GetAccounts/GetAccount to handle the user
- [x] AccountType : Binance/EvmTrador/EvmWatch/EvmPersonal
- [x] Make better abstration of the Database projects (influx, mongodb) to Remove the link between Application and Infrastructure
- [x] Implement strategy pattern for exchange processors
- [x] Use cache for position and signal
- [x] Don't run update Signal and manage position if no new candle
### Workers
- [x] Add Influxdb
- [x] Create workers for scrap data (per ticker, per exchange, per timeframe)
- [x] Use candle from influxdb for backtests and bots
- [x] Create a worker that get a daily top 15 of ticker volume
- [x] Scrap candle only for crypto with high volume
- [x] Create a worker that run backtest and save it in db
### Front-end
- [x] Return list of balances with tokens/chains data
- [x] On account line click, display tokens by chain available
- [x] Call Available ticker to produce the droplist
- [x] Display multipanes https://github.com/qwpto/lightweight-charts-panels / https://codesandbox.io/s/react-typescript-forked-mxek17?file=/src/App.tsx / https://github.com/Rassibassi/multipane_tradingview
- [x] Display last open position with lines (add SL and TP)
- [x] Fix double charts problem (has been fixed by removing strict mode)
- [x] Try to pass the tailwind current theme color to the chart
- [x] List all open position for every trading account
- [x] Create button to force close position
- [x] Open Position button
- [x] Improve position message -> Create message buildor
- [x] Create channel handler for address watcher
- [x] Add input to specify the account balance. If not set, it will use the selected account balance
# The rest
## Part 1 : Adaptive trading
- [x] From backtests list : Determine TP and Stop loss % by getting average profit and average lost for a signal
- [x] Modify bot to use MoneyManagement object instead of using the risk associated to the signal.
- [x] Bot object should return MoneyManagement object
- [ ] Strategy can return a suggested trade that will give the best orders to execute
## Part 2 : Workflows builder
- [x] Create a workflow type that will encapsulate a list of flows
- [x] Each flow will have parameters, inputs and outputs that will be used by the children flows
- [ ] Flow can handle multiple parents
- [ ] Run Simple bot base on a workflow
- [ ] Run backtest based on a workflow
- [ ] Add flows : ClosePosition, Scenario
## Part 2 : Strategies optimisation
- [ ] Optimize all strategies with more data (funding rate trend, bbwp, etc..)
- [ ] Optimize entry by getting last ask price and place an order limit to match directly with it
- [ ] Add bbwp indicator => PR example : https://github.com/DaveSkender/Stock.Indicators/pull/893/files#diff-df1cd2a99846f7328c9fc3db4d2c164380a0fa943cdcad2ece01b886cac68137
- [ ] Create strategies types from signal (a divergence) / trend (stoch long or short) / context (high vol/low vol)
- [ ] Create a trend strategy on fundinrate -> long trend if negativ fr / short trend if positiv fr
- [ ] Add entry/exit price to signal for adaptive risk
- [ ] MACD: Open only when candle is not a big move to avoid late entry (only enter when low vol)
- [ ] Improve strategy composability by using delegate pattern
## Part 3 : Backtests automation
- [ ] Create a worker that determine best TP and SL
- [ ] Create a page 'Analyze' to show the best TP and SL for multiple backtest result
- [ ] Implement reverseSignal to close and flip the trade
- [ ] Load signal from database
- [ ] Create a bot to update signal into database that still in pending
- [ ] Signal return best SL and TP
- [ ] Combine multi-timeframe to detect better signals => Adapt risk on the lowest timeframe based on upper timeframe
- [ ] Remove candles from the backtestResult
- [ ] Save start/end time and timeframe into the result
- [ ] On GET backtest -> Retrieve the candle from cache or database
_______________________________________________________________________________________________________________
# Phase 2 - Web3
## 1 - Wallet management
- [ ] User can deposit token to and evm account with a QR code or clicking on button to copy the address
## 2 - [Track address](https://github.com/users/CryptoOda/projects/1/views/1?pane=issue&itemId=32158941)
- [ ] Create a snapshot of the account balances with BalancesWorker
- [ ] Display balance in $ evolution per token hold on the account page
## 3 - Settings [Link](https://github.com/users/CryptoOda/projects/1/views/1?pane=issue&itemId=32159287)
- [ ] Save theme into database
- [ ] Call GET Settings/Theme to retrieve the theme used by the user
## 4 - Trading desk [Link](https://github.com/users/CryptoOda/projects/1/views/1?pane=issue&itemId=11070087)
- [x] Live data position hub
- [x] Live candles
- [x] Open position from desk
## 5 - Live liquidation map
- [ ] Create worker to aggregate binance/okx liquidation
- [ ] Render liquidation map with a hub subscription to get live update
- [ ] Live data signal
- [ ] Display massive liquidation on TradeChartWidget
## 6 - Metrics [Link](https://github.com/users/CryptoOda/projects/1/views/1?pane=issue&itemId=32159107)
- [ ] Create a worker that get top 5 cryptos by scenario
- [ ] Create worker to store portofolio evolution
- [ ] Display portofolio evolution
- [ ] Create a worker to scrap funding fees over time (per ticker, per exchange)
## 7 - Improvement [Link](https://github.com/users/CryptoOda/projects/1/views/1?pane=issue&itemId=32159127)
- [ ] Handling user on every object => Use the JWT token to inject the userName
- [ ] Extract BacktestReport from tests to a Tools projects
- [ ] Factorize amount management for GMX Service
- [ ] Remove link betweend Domain object to EVM project
- [ ] GMX Service should handle only Nethereum object, not Domain.Models
- [ ] Compute backtests pagination on backend side
- [ ] Store webapp into docker
- [ ] Make possible to force close all positionpositions of a bot
- [ ] Save bot config to easily relaunch a bot and also analyze data
- [ ] Bot can handle limit order position -> fetch if order is still open then check if position of open
- [ ] Create method to update the money management use by the bot
- [ ] Implement from/to tickers array pattern
- [ ] Extract all managing trade method into a TradingBox class => Create composable trading bot type easily
# Front-end
## Improve Account page
- [ ] Display total balances (only trader account) on Settings->Accounts Page
- [ ] On Token Account details line -> Add button to withdraw money (this will open a Modal to prompt the address/amount that will receive the money)
## Charts
- [ ] Add multi-timeframe options
- [ ] Display optionnal indicators on charts
- [ ] Display current selected position (when i click on a Position in the list)
## Trading desk
- [ ] WAIT - Create a button to open a hedge on option market
- [ ] Get global exposure ((total position / total balances *) 100)
- [ ] Display multiple tab to show different tickers
## Scenarios
- [x] Create a blocky-like editor to create scenarios https://google.github.io/blockly-samples/
## Bots
- [ ] Add button to display money management use by the bot
- [ ] On the modal, When simple bot selected, show only a select for the workflow
## Backtests
- [x] Display the money management used by the backtest
## Technical front-end :
- [ ] Implement alias to reduce 'import' lines https://github.com/subwaytime/vite-aliases
- [ ] Make Get Accounts List as a store
- [ ] Replace all html-tag Modals (ex: BacktestPlayground.tsx)
- [ ] Refactoring Components :
- organism : Put all the code related to the app (Trade charts etc). This folder should be delete when the project is forked
- mollecules : Put aggregate of atoms to create forms/modal/table
- atoms : Put all basic html tag components (List, Select, Slider...)
## Workflow
- [x] List all workflow saved in
- [x] Use https://reactflow.dev/ to display a workflow (map flow to nodes and children to edges)
- [x] On update Nodes : https://codesandbox.io/s/dank-waterfall-8jfcf4?file=/src/App.js
- [x] Save workflow
- [ ] Reset workflow
- [ ] Add flows : Close Position, SendMessage
- [ ] On Flow.tsx : Display inputs/outputs names on the node
- [ ] Setup file tree UI for available flows : https://codesandbox.io/s/nlzui
_____________________________________________________________________________________________________________
# Technicals
- [ ] Remove mediator
- [ ] Check Traefik : https://www.cloudbees.com/blog/setting-up-traefik-as-a-reverse-proxy-for-asp-net-applications
- [ ] Implement IHostedService/BackgroundService to run bot in background https://www.youtube.com/watch?v=fw-n2RkzOMI
- [ ] Implement Microsoft Orleans

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,12 @@
```mermaid
flowchart TD
A[Futures bot] -->|50% weekly withdraw| B[Delta neutral bot]
A -->|25% weekly withdraw| F[Stake GMX]
A -->|25% weekly withdraw| E
B -->|25% profit | A
B -->|75% compounding| B
D[Hedging bot] -->|hedge open position| A
D -->|generate profit| A
E[Staking bot] -->|claim ETH rewards| F[Personnal Wallet]
```

View File

@@ -0,0 +1,27 @@
```mermaid
classDiagram
Workflow <|-- Flow
class Workflow{
String Name
Usage Usage : Trading|Task
Flow[] Flows
String Description
}
class Flow{
String Name
CategoryType Category
FlowType Type
FlowParameters Parameters
String Description
FlowType? AcceptedInput
OutputType[]? Outputs
Flow[]? ChildrenFlow
Flow? ParentFlow
Output? Output : Signal|Text|Candles
MapInput(AcceptedInput, ParentFlow.Output)
Run(ParentFlow.Output)
LoadChildren()
ExecuteChildren()
}
```

View File

@@ -0,0 +1,25 @@
```mermaid
graph LR
    subgraph Position
        A(New)
        F
        G(Filled) --> L(Flipped)
        G --> H(Finished)
        A --> I(Cancelled)
        A --> J(Rejected)
    end
    subgraph Trade Open
        A --> N
        N(Requested) --> O(Filled)
        O --> F(Partilly Filled)
    end
    subgraph Trades SL/TP
        F --> B(PendingOpen)  
        B -->C(Requested)
        C -->P(Cancelled)
        C -->D[Filled]
        D --> G
    end
```

BIN
assets/img/Architecture.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 199 KiB

BIN
assets/img/doc-docker.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 41 KiB

BIN
assets/img/doc-influxdb.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

BIN
assets/img/doc-settings.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.3 KiB