SBA Funds Orchestration
About This Page
What: What SBA is, the Procedure data model and key fields, full state machines for Deposit and Withdrawal (with Operations focus), detailed BST Deposit/Withdrawal Orchestration flows, SBA service API overview, async task queues and polling, deduplication and idempotency, Freeze-Transfer-Release pattern, Procedure lookup and troubleshooting guide, and common exception scenarios with resolution steps Audience: Product managers and Operations staff who need to understand internal funds Orchestration logic Prerequisites: System Architecture & Data FlowReading time: 12 minutes Owner: Deposit & Withdrawal Product Team
Key takeaway: SBA is the central Orchestration system for all funds operations — every Deposit/Withdrawal creates a Procedure record that follows a standard "Freeze → Transfer → Release" flow. Understanding the Procedure state machine is the foundation for troubleshooting funds exceptions.
What Is a Procedure
Every Deposit or Withdrawal creates a Procedure record in the SBA Orchestration system. It is the complete tracking entity for a funds operation from creation to completion (or failure/cancellation) — think of it as a "funds operation work order."
Relationship Between Procedure and Business Layer
Procedures don't appear out of thin air — they are created by the business layer (Deposit service / Withdrawal service) at specific points:
| Business Object | SBA Object | Creation Trigger | Association |
|---|---|---|---|
Apply (Deposit application) | ProcedureCashDeposit | Bank Statement Matching succeeds + auto-crediting conditions met | sba_procedure_id field in the Apply record |
Task (Withdrawal task) | ProcedureCashWithdraw | All three Withdrawal Approval steps pass (Remittance stage) | sba_list association table (Task ID ↔ Procedure ID) |
PM perspective: Users see Apply (Deposit) or Task (Withdrawal) in the App, and Operations see the same in CRM. Procedure is a "behind-the-scenes actor" — you only need to examine it when investigating funds flow details. But when "money didn't arrive" or "money was deducted but not transferred out," Procedure is the key to pinpointing the issue.
Key Fields
| Field | Business Meaning | Common Operations Use |
|---|---|---|
id | Procedure unique ID, globally incrementing | Primary key for CRM search and troubleshooting |
biz_type | Business type: cash_deposit (Deposit) or cash_withdraw (Withdrawal) | Distinguish between Deposit/Withdrawal Procedures |
nn_uid | User ID (NiuNiu ID) | Link to a specific user |
market | Trading market: 1=HK, 2=US, 4=A-Share Connect | BST requires separate Procedures per market |
trade_acc_id | Trading account ID | A user may have multiple trading accounts |
status | Main status — indicates which major stage of the lifecycle the Procedure is in | Status filter on CRM list page |
ext_status | Sub-status — finer-grained tracking within the same main status | Key to locating "which step it's stuck at" |
version | Optimistic lock version number, increments by 1 on each update, prevents concurrent write overwrites | Refresh and retry when CRM shows "version conflict" error |
transaction_date | Settlement date (trading day format YYYYMMDD), determines which trading day the credit belongs to | Root cause of cross-day Withdrawal delays |
reason | Terminal reason — only populated on failure/rejection/cancellation | Investigating "why did this Withdrawal fail" |
Version Conflict Handling
CRM Operations staff occasionally encounter a "version conflict" error when operating on a Procedure. This is the optimistic lock mechanism — it means the system (scheduled tasks, bank callbacks, etc.) was also updating this Procedure while you were operating on it.
Resolution: Refresh the page to get the latest version, then retry. If conflicts persist (>3 times), concurrent tasks are frequently updating — wait 10–30 seconds and try again. Never bypass the version check by modifying the database directly.
Audit Trail (Flow)
Every time a Procedure undergoes a status change, SBA writes a Flow record (audit log). The operation history shown when clicking "View History" in CRM comes from this table.
| Field | Meaning | Operations Use |
|---|---|---|
procedure_id | Associated Procedure ID | Query full change history by Procedure ID |
from_status / to_status | Main status before and after the change | Verify whether the status transition is expected |
from_ext_status / to_ext_status | Sub-status before and after the change | Pinpoint exact change details |
operator | Who performed it (system = automatic / staff_xxxx = CRM Operations) | Distinguish system actions from manual actions |
remark | Notes | Reason provided by Operations during manual actions |
created_at | Change timestamp (millisecond precision) | Timeline for issue investigation |
Flow records cannot be modified or deleted — they are the complete audit trail for Deposit and Withdrawal operations.
Troubleshooting tip: When a Procedure is in an abnormal state, view Flow records sorted by created_at descending and find the last change — the pre-change status, operator, and timestamp often directly pinpoint the root cause.
Full Status Code Reference
All Procedure main status + sub-status combinations → Withdrawal Data Dictionary § SBA Procedure Status Codes
Deposit Procedure State Machine
The Deposit state machine is relatively simple (6 states), because the complexity lies in the Matching engine — once Matching is complete, the Procedure only handles the final crediting step.
Status Descriptions and Operations Focus
| Main Status | Sub-Status | Business Meaning | Terminal? | Operations Focus |
|---|---|---|---|---|
new | waiting | Procedure created, awaiting automatic/manual decision | No | Normal state, system processes within seconds. If it lingers (>1 minute), check if SBA service is healthy |
new | manual_confirm | Triggered high-risk or specific rules, requires CRM Operations review | No | Action required — process in the CRM Deposit review queue. Before reviewing, confirm: is the amount reasonable, is the user on the blacklist, does the Bank Statement match |
pending | (empty) | Review passed, executing Risk Control asset change (adding funds) | No | Transitional state, usually completes in milliseconds. If stuck, the Risk Control system may be experiencing issues |
end_ok | deposit_ok | Deposit successful, funds credited to Securities Account | Yes | Normal terminal state. User can see the balance increase in the App |
end_reject | (empty) | Rejected — CRM review denied or Risk Control asset change failed | Yes | Check the reason field for the rejection reason. If it's a Risk Control change failure (not a manual rejection), escalate to technical team |
end_reverse | deposit_ok | Credited funds have been reversed (withdrawn back) | Yes | High-risk operation — only perform when confirming a Deposit error (e.g., duplicate crediting, fraud). User balance will decrease after Reversal |
Scenarios That Trigger Manual Review
The following situations cause a Deposit Procedure to enter manual_confirm, requiring CRM Operations handling:
| # | Scenario | Decision Rule | Operations Handling |
|---|---|---|---|
| 1 | High-risk Deposit | deposit_type = HIGH_RISK(5) | Verify user identity and source of funds; approve after confirming no risk |
| 2 | Abnormal Deposit | deposit_type = ABNORMAL(3) | Check Bank Statement details; manually match after confirming amount and source |
| 3 | Pre-Approval Deposit | deposit_type = PRE_APPROVAL(2) | Verify whether pre-Approval conditions are met |
| 4 | Blacklist hit | Deposit blacklist match | Confirm whether it's a false positive; reject if not |
| 5 | Large Deposit | Exceeds auto-crediting threshold | Approve after verifying source legitimacy |
BST Deposit Orchestration Detailed Flow
BST Deposit skips the Matching engine — the bank pushes results directly or they are obtained via polling, and SBA directly creates a Procedure and credits the account. Below is the complete system flow for Airstar BST Deposit:
CMB/CMBC BST Deposit differs in Phase 2: no polling is needed — the bank pushes results in real time via an SM2 Socket bidirectional link, reducing latency from "seconds to minutes" down to "seconds."
BST Deposit requires that the Bank Card has completed BST authorization (Mandate status = OPEN). For the authorization flow and Bank Card management → Bank Cards & Authorization § BST Authorization
Deposit Extended Fields (ProcedureCashDeposit)
| Field | Meaning | Operations Use |
|---|---|---|
transaction_id | Non-BST: Bank Statement ID; BST: business ID | Key identifier for reconciliation with the bank |
amount | Deposit amount | Confirm amount is reasonable during review |
ccy | Currency (HKD / USD / CNH) | Confirm currency matches the Bank Statement |
is_bs_transfer | 0=non-BST; 1=BST Channel | Distinguish BST Deposit from regular Deposit processing logic |
transfer_method | TransType code (101/102/201-304, etc.) | Identifies the source bank and Channel |
deposit_type | Deposit mode: NORMAL(1) / PRE_APPROVAL(2) / ABNORMAL(3) / TRANS_AUTO(4) / HIGH_RISK(5) / NORMAL_HOLD(11) | Determines whether manual review is required |
hold_type | Freeze type (NORMAL_HOLD mode only): 1=EXCHANGE / 2=CASH_OUT / 17=CASH_IN | Purpose of funds Freeze |
apply_id | Freeze ID (32-character unique identifier, required for NORMAL_HOLD mode) | Links to Freeze record |
source | Deposit source: 1=initiated by moomoo; 2=initiated by bank | Bank-initiated Deposits processed via AsbBstCreateFromBankJob |
bank_ref_id | Bank-side reference number (ASBBST + sequence) | Unique identifier for bank reconciliation |
Full TransType table → Deposit Quick Reference § TransType
The Deposit Procedure state is relatively simple — the complexity lies in the Matching engine, and the Procedure only handles the final crediting. The Withdrawal Procedure that follows is far more complex, as it must handle Freeze, deduction, bank transfer, and failure rollback.
Withdrawal Procedure State Machine
The Withdrawal state machine is the most complex in the entire system (17 states), because it must handle Freeze, deduction, transfer, failure rollback, cross-trading-day delays, and many other scenarios.
Three-Phase Overview
| Phase | Statuses Involved | What Happens | Key PM Question | User Perception |
|---|---|---|---|---|
| 1. Freeze | new(freeze) → new(waiting) | System Freezes the user's available balance | "Has the user's balance decreased?" | Available balance decreases, but total assets remain the same (frozen amount is still in the account) |
| 2. Review | new(waiting) → new(manual_confirm) → new(confirmed) | Automatic/manual review | "Can the user cancel?" | Can cancel during Freeze stage; cannot cancel once deduction begins after review |
| 3. Deduction + Transfer | new(blank) → pending(deduct_done) → pending(transfer_auto/manual) | Balance formally deducted, transfer instruction sent to bank | "Money deducted but not at the bank?" | Balance deducted, waiting for bank processing |
| 4. Result | end_ok / end_reject / end_cancel | Bank confirms success or failure | "When will the user receive the money?" | end_ok = bank confirmed, funds remitted |
Happy Path
Most Withdrawals follow this path — Freeze → Deduction → Transfer → Success:
Beyond the happy path, Withdrawals may encounter failure, cancellation, cross-trading-day delays, and other exception branches — these paths deserve the most attention when troubleshooting:
Exception and Rollback Paths
Key Status Descriptions and Operations Focus
| Main Status | Sub-Status | Meaning | Terminal? | Operations Focus |
|---|---|---|---|---|
new | freeze | Initial state — waiting to Freeze user balance | No | Completes in milliseconds. If it stays for >30 seconds, check if Risk Control system is responding |
new | waiting | Freeze successful, waiting for scheduled script or manual processing | No | Regular Withdrawals are automatically picked up by scheduled scripts; BST Withdrawals auto-enter deduction. If lingering, check scheduled tasks |
new | manual_confirm | Requires CRM manual review | No | Action required — Withdrawal review queue. Before reviewing, confirm: Withdrawal amount vs. available balance, target Bank Card belongs to the user, whether Risk Control rules were triggered |
new | confirmed | Manual review passed, pending deduction | No | Transitional state. System automatically enters the deduction flow |
new | (empty) | Pending deduction (new_blank) | No | Deduction in progress. If stuck for >1 minute, the Risk Control asset change API may have timed out |
pending | deduct_done | Deduction complete, pending transfer method confirmation | No | Critical checkpoint — deduction is complete, cannot roll back to Freeze. BST auto-enters transfer; online banking/FPS enters manual transfer |
pending | transfer_auto | BST auto-transfer in progress | No | BST Channel is sending the transfer instruction to the bank. Airstar requires polling for results (up to 10 times); CMB/CMBC push in real time |
pending | transfer_manual | Manual transfer in progress (online banking / FPS / wire, etc.) | No | Action required — after completing the transfer in the bank's online banking portal, click "Confirm Transfer Complete" (SetManualTransferDone) in CRM |
pending | return | Transfer failed, pending funds restoration | No | System automatically restores the deducted amount to the user's account. If restoration fails, critical alert — user's money has "disappeared" |
pending | unfreeze | Withdrawal cancelled, unfreezing in progress | No | System automatically releases the frozen amount. If unfreezing fails, the user's balance will be "stuck" |
pending | move_next | Non-trading hours, deferred to next trading day | No | Triggered when BST Withdrawal is submitted outside trading hours. Procedure returns to new(freeze) and restarts the flow |
pending | waiting_trade | Waiting for trading day to arrive | No | Withdrawals submitted on weekends/holidays wait here. Automatically enters terminal state when trading day arrives |
pending | transfer_reject | Rejected/cancelled, awaiting terminal state confirmation | No | Transitional state. System automatically sets the final reject or cancel |
end_ok | transfer_done | Withdrawal successful | Yes | Bank confirmed remittance successful. User should receive funds within 1–3 business days |
end_reject | (empty) | Withdrawal failed | Yes | Check the reason field. Funds have been automatically restored to the user's account |
end_cancel | (empty) | Withdrawal cancelled | Yes | User-initiated or Operations-initiated cancellation. Funds have been automatically unfrozen |
end_reverse | transfer_done | Completed Withdrawal has been reversed | Yes | Extremely high-risk operation — only use when the bank confirms remittance failure/return |
pending_move Loop Mechanism
pending_move is a unique looping state: BST Withdrawals submitted outside trading hours are moved to the next trading day by the MoveNextTransactionDay operation — the Procedure returns to new(freeze) and restarts the Freeze-Deduction-Transfer flow.
| Scenario | Example | Procedure Behavior | User Perception |
|---|---|---|---|
| BST Withdrawal submitted at 18:00 on Friday | Trading hours have ended | pending_move → next Monday new(freeze) | Submitted Friday, processed Monday |
| Submitted the day before a holiday | Same logic | Deferred to the first trading day after the holiday | Processed on the first trading day after the holiday |
| Submitted after 16:30 on a trading day | Past the daily cutoff time | Deferred to next day new(freeze) | Processed next day |
| Consecutive holidays (e.g., Chinese New Year) | Friday → next Tuesday is the first trading day | May loop through pending_move multiple times | Processed after the holiday ends |
Common User Question
"Why was my Friday Withdrawal not received until Monday?" — Because BST Channels depend on trading days, and Withdrawals submitted outside trading hours are deferred to the next trading day. This is not a system error; it is normal business logic.
Suggested Operations response: "Your Withdrawal has been accepted. Since BST Channels process during trading hours, it will be automatically processed on the next trading day (Monday), with an estimated arrival in 1–2 business days."
Trading Day Rules
Rules the system uses to determine "whether we are currently in trading hours":
| Market | Trading Days | Cutoff Time | After Cutoff |
|---|---|---|---|
| HK (Hong Kong equities) | Monday–Friday, excluding HK public holidays | 16:15 HKT | Withdrawal deferred to next trading day |
| US (US equities) | Monday–Friday, excluding US public holidays | 04:00 HKT next day (corresponding to 16:00 ET) | Withdrawal deferred to next trading day |
| HKCC (Stock Connect) | Stock Connect trading days (excluding both HK and Shanghai/Shenzhen holidays) | 16:15 HKT | Withdrawal deferred to next trading day |
Public holiday source: The system maintains a trading calendar (updated by the settlement team at the end of each year), covering HKEx, NYSE, and SSE/SZSE market closures. The transaction_date field is automatically calculated by SBA based on the current time and the trading calendar.
Troubleshooting tip: If a Withdrawal triggers pending_move during daytime on a business day, check whether the trading calendar includes that day (it may be a temporary closure or the trading calendar may not have been updated).
BST Withdrawal Orchestration Detailed Flow
Below is the complete system flow for Airstar BST Withdrawal from user initiation to funds arrival:
Online banking / FPS Withdrawal differs in Phase 3: instead of auto-transfer, it enters pending(transfer_manual) and waits for Operations to complete the transfer manually in the bank's corporate online banking portal before confirming.
Withdrawal Extended Fields (ProcedureCashWithdraw)
| Field | Meaning | Operations Use |
|---|---|---|
amount | Withdrawal amount | Verify amount is reasonable during review |
fee | Transaction fee (currently all 0) | — |
ccy | Currency | Confirm currency matches user's selection |
transfer_method | Channel key: auto_bs / hase / hsbc / boc_fps / cgb_fps_api, etc. | Determine which Withdrawal Channel is used |
transfer_dest_bank | Target bank code | Verify target bank information |
transfer_dest_card | Target Bank Card number | Verify receiving account number |
bank_launched | 0=initiated by moomoo; 1=initiated by bank (BST-specific) | Distinguish who initiated this Withdrawal |
transfer_ref_id | Bank-side reference number (CMB=bank_tx_seq, CMBC=OrgRefNo, Airstar=ASBBST+sequence) | Key identifier for bank reconciliation |
settled_amount | Cash portion amount | For margin account Withdrawals, distinguishes cash vs. margin portion |
margin_amount | Margin (loan) portion amount | Monitor during margin Withdrawals |
payee_name | Payee name (CMBC BST uses English name) | Verify payee identity |
request_id | Bank-side request ID (Airstar) | Primary key for polling results |
Full Channel code table → Withdrawal Data Dictionary § Withdrawal Channel Codes
Now that you understand the state machine, the next step is to learn what operations Operations staff can perform on Procedures in CRM — each CRM button corresponds to an SBA API.
SBA Service API Overview
These are the operation APIs exposed by the SBA Orchestration system — they correspond to buttons in CRM. Understanding "which button calls which API" helps troubleshoot "why a button click had no effect."
Deposit Procedure Operations
| Operation | Who Triggers | Effect | Prerequisite Status | CRM Mapping | Permission Required |
|---|---|---|---|---|---|
CashDepositCreate | Deposit service (automatic) | Creates a Deposit Procedure, starts crediting flow | — | Auto-triggered, no CRM button | System automatic |
CashDepositManualConfirm | CRM Operations | Approves review, continues crediting | new(manual_confirm) | Deposit Review → "Approve" | Frontline Operations |
CashDepositManualReject | CRM Operations | Rejects review, Procedure → end_reject | new(manual_confirm) | Deposit Review → "Reject" | Frontline Operations |
CashDepositReverse | CRM Operations | Reverses a completed Deposit, funds withdrawn from Securities Account | end_ok | Deposit Details → "Reverse" | Supervisor level |
Reversal Operation Warning
CashDepositReverse directly deducts the credited amount from the user's Securities Account. Before executing, you must confirm:
- It is genuinely an erroneous credit — e.g., duplicate crediting, fraud, bank refund
- The user's current available balance is sufficient for deduction — if the user has already used the deposited funds for trading or Withdrawal, insufficient available balance will cause the Reversal to fail
- The user has been notified — the user's balance will decrease after Reversal, so prior communication is required
Reversals are irreversible. An erroneous Reversal requires re-initiating the Deposit flow.
Withdrawal Procedure Operations
| Operation | Who Triggers | Effect | Prerequisite Status | CRM Mapping | Permission Required |
|---|---|---|---|---|---|
CashWithdrawCreate | Withdrawal service (automatic) | Creates a Withdrawal Procedure, starts Freeze | — | Auto-triggered | System automatic |
CashWithdrawStartTransfer | CRM Operations / automatic | Confirms transfer parameters are correct, starts transfer | new(blank) or pending(deduct_done) | "Start Transfer" | Frontline Operations |
CashWithdrawReject | CRM Operations | Rejects Withdrawal, triggers unfreeze rollback | Any status in the new phase | "Reject" | Frontline Operations |
CashWithdrawManualConfirm | CRM Operations | Manual review approved | new(manual_confirm) | "Approve" | Frontline Operations |
CashWithdrawManualReject | CRM Operations | Manual review rejected | new(manual_confirm) | "Reject Review" | Frontline Operations |
SetManualTransferDone | CRM Operations | Confirms manual transfer (online banking / wire, etc.) is complete | pending(transfer_manual) | "Confirm Transfer Complete" | Frontline Operations |
SetManualTransferFailed | CRM Operations | Confirms manual transfer failed, triggers funds restoration | pending(transfer_manual) | "Transfer Failed" | Frontline Operations |
CashWithdrawForceProcess | CRM Operations | Force-processes a stuck Procedure | Any non-terminal status | "Force Process" | Supervisor level |
CashWithdrawCancel | CRM Operations | Cancels Withdrawal, Procedure → end_cancel | new phase (after Freeze, before deduction) | "Cancel Withdrawal" | Supervisor level |
MoveNextTransactionDay | CRM Operations / scheduled task | Defers processing to the next trading day | pending(deduct_done) / pending(transfer_manual) | "Defer to Next Trading Day" | Frontline Operations |
CashWithdrawReverse | CRM Operations | Reverses a completed Withdrawal | end_ok | "Reverse" | Supervisor level |
Button Not Available?
Whether a CRM button is clickable depends on the Procedure's current status. For example, "Approve" is only available in the new(manual_confirm) state — if the Procedure has already entered the pending phase, the button will be grayed out.
Troubleshooting "button click had no effect":
- Check whether the Procedure's current
status+ext_statusmatch the operation's prerequisite status - Check whether
versionis up to date — refresh the page and retry - Check operator permissions — certain operations (e.g., Reversal) require specific roles
CRM operations ultimately trigger SBA's interactions with the bank. But banks don't respond synchronously — next, let's look at how SBA manages bank interactions through async task queues.
Async Task Queue Pattern
Most interactions between the Deposit/Withdrawal system and banks are asynchronous — after sending an instruction, you don't get the result immediately. SBA manages these async flows through task queues.
Execution Pattern
- Create Job: Receives the business request, assembles parameters, sends instruction to the bank
- Result Job: Periodically polls the bank API to query processing results
- Retry mechanism: Has an explicit maximum retry count; exceeding it marks the job as an exception and triggers an alert
- Idempotency: Uses
procedure_id+request_idto ensure the same instruction is not executed twice
Airstar BST Task Chain Details
Airstar BST is a typical async polling pattern — each operation consists of a creation task and a result-polling task:
Deposit task chain:
| Task Name | Max Retries | Poll Interval | Responsibility | Failure Handling |
|---|---|---|---|---|
AsbBstCreateJob | — | — | Send Deposit request to Airstar, obtain request_id | Mark as failed, notify user |
AsbBstCreateFromBankJob | — | — | Process bank-initiated Deposits (source=2) | Mark as exception, pending manual verification |
AsbBstCreateResultJob | 60 | ~3 seconds each | Poll bank for Deposit result — the key factor determining crediting speed | 60 polls with no result → Feishu alert + manual intervention |
AsbBstCreateRetryJob | — | — | Retry temporarily failed Deposit requests | Retry failure → mark as terminal failure |
AsbBstDepositJob | — | — | After Deposit succeeds, execute SBA accounting (CashDepositCreate) | Accounting failure → critical alert |
Withdrawal task chain:
| Task Name | Max Retries | Poll Interval | Responsibility | Failure Handling |
|---|---|---|---|---|
AsbBstTransfer | 10 | ~5 seconds each | Send Withdrawal request and poll for result | 10 polls with no result → Feishu alert |
SyncAsbBstWithdraw | — | Every 2 hours | Periodically pull bank-initiated Withdrawal requests | Pull failure → retry |
Bank-initiated transactions (source=2) processing logic:
| Direction | Pull Task | Pull Window | Processing Method |
|---|---|---|---|
| Deposit | FetchBankRequest | Every 2 minutes | After pulling, creates AsbBstCreateFromBankJob for processing |
| Withdrawal | SyncAsbBstWithdraw | Every 2 hours | After pulling, compares against local records; creates Withdrawal Procedures for new entries |
Polling Timeout Troubleshooting
When a user reports "Deposit/Withdrawal is stuck in processing," follow these steps:
- Check authorization status: Is the Mandate OPEN (status=2)? If not → re-authorization needed
- Check retry count: View
retry_countin the Job record — Deposit max 60 (about 3 minutes), Withdrawal max 10 (about 50 seconds) - Check bank response: If the Job record contains a bank error code, use that to pinpoint the issue
- Retries exhausted with no result: Contact Airstar Bank to verify the bank-side status
- Bank-initiated transactions (source=2): Check whether
FetchBankRequest/SyncAsbBstWithdrawpull tasks are running normally
CMB/CMBC SM2 Socket Tasks
CMB and CMBC do not use polling — they communicate in real time via SM2-encrypted Socket bidirectional links:
| Dimension | CMB (cmb_stock_trans) | CMBC (ms_stock_bank_transaction) |
|---|---|---|
| Connection | SM2 Socket persistent connection | SM2 Socket persistent connection |
| Deposit | Bank pushes → received in real time | Bank pushes → received in real time |
| Withdrawal | moomoo sends → bank pushes result | moomoo sends → bank pushes result |
| Heartbeat | Scheduled heartbeat keepalive | Scheduled heartbeat keepalive |
| Disconnection recovery | sync_bst_data_helper compensation | Auto-reconnect + re-pull unprocessed records |
| Typical latency | Sub-second | Sub-second |
| Operations intervention probability | Low (unless Socket disconnects) | Low (unless Socket disconnects) |
Async Mode Comparison Across Banks
| Bank | Communication Method | Polling Required? | User Wait Time | Operations Intervention Probability |
|---|---|---|---|---|
| Airstar BST | REST API (JSON) | Yes (Deposit 60 / Withdrawal 10) | Seconds to minutes | Medium |
| CMB BST | SM2 Socket bidirectional link | No (real-time push) | Sub-second | Low |
| CMBC BST | SM2 Socket bidirectional link | No (real-time push) | Sub-second | Low |
| HSBC eDDA | REST API | Yes | T+0 to T+1 | Medium |
| Hang Seng eDDA | REST API | Yes | T+0 to T+1 | Medium |
PM perspective: Banks that require polling result in longer user wait times and higher probability of Operations intervention during exceptions. This is why Airstar BST, though also "fully automated," may take longer from submission to crediting compared to CMB/CMBC.
Specific Job names and retry counts → BST Overview § Scheduled Tasks
Deduplication and Idempotency
The biggest fear in funds operations is "duplication" — a Deposit credited twice, a Withdrawal deducted twice. SBA establishes deduplication safeguards at multiple layers.
Three-Layer Deduplication
| Layer | Mechanism | Deduplication Key | What It Prevents |
|---|---|---|---|
| 1. Business layer | Apply/Task uniqueness | Deposit: flow_id + bank_type; Withdrawal: task_id | Prevents duplicate Procedure creation from the same Bank Statement/task |
| 2. SBA layer | Procedure creation idempotency | procedure_id + request_id | Prevents duplicate creation from SRPC timeout retries |
| 3. Bank layer | Bank-side reference number | Airstar: ASBBST prefix + sequence; CMB: bank_tx_seq; CMBC: OrgRefNo | Prevents duplicate transfer instructions sent to the bank |
Airstar Deduplication Example
Airstar BST's bank_ref_id uses an ASBBST prefix + globally incrementing sequence (e.g., ASBBST000012345), ensuring:
- Deposit: The same
request_idwill not result in duplicate crediting — even ifAsbBstDepositJobis executed repeatedly - Withdrawal: The same
bank_ref_idwill not result in duplicate transfer instructions to the bank — even ifAsbBstTransferretries on timeout
Concurrency Control
| Mechanism | Implementation | Scenario |
|---|---|---|
| Optimistic lock | Procedure's version field, update with WHERE version = ? | Prevents CRM Operations and system scheduled tasks from simultaneously updating the same Procedure |
| Distributed lock | Redis lock, key = procedure:{id} | Prevents the same Procedure from being processed concurrently by multiple Workers |
| Idempotency check | Deposit: check if transaction_id already exists; Withdrawal: check if a Procedure for task_id already exists | Prevents duplicate creation |
Deduplication ensures "no extra operations." The Freeze pattern that follows ensures "no errors" — until the bank confirms success, the user's funds are always safely locked.
Freeze-Transfer-Release Pattern
Withdrawal is not as simple as "just transfer the money out." SBA uses a Freeze-Transfer-Release three-step pattern to ensure funds safety.
Why Freeze First
Between the time the system sends a Withdrawal instruction and the bank confirms success, there is a time window (ranging from seconds to hours). If funds are not frozen during this window:
- The user could simultaneously initiate another Withdrawal or trade
- The account could end up with a negative balance
- Funds contention could occur
Freezing "reserves" the funds during this time window, ensuring they are not used by other operations.
Impact of Freeze on User Assets
| Asset Metric | Before Freeze | After Freeze (Withdrawal in progress) | Withdrawal Succeeds | Withdrawal Fails (Unfrozen) |
|---|---|---|---|---|
| Total assets | 100,000 | 100,000 (unchanged) | 90,000 (decreased) | 100,000 (restored) |
| Available balance | 100,000 | 90,000 (decreased by 10,000) | 90,000 | 100,000 (restored) |
| Frozen amount | 0 | 10,000 | 0 | 0 |
During Withdrawal processing, the user sees "available balance" decrease while "total assets" remain unchanged — this is normal Freeze behavior.
Three Outcomes
| Outcome | Freeze Handling | User Perception | Operations Action |
|---|---|---|---|
| Success | Freeze released, funds formally deducted | Balance decreases, user receives arrival notification | No action needed |
| Failure | Freeze released, funds restored to available | Balance restored, user receives Withdrawal failure notification | Investigate failure reason, notify user if necessary |
| Timeout | Freeze maintained, awaiting confirmation | Balance not restored, user may contact customer service | Must manually confirm bank-side status before deciding to release or deduct |
Timeout Handling Flow
Timeout is the trickiest scenario — funds may have already reached the user's Bank Card (bank-side success), but moomoo hasn't received confirmation yet:
| Step | Action | Description | Timeout Threshold |
|---|---|---|---|
| 1 | Receive timeout alert | Feishu notifies Operations | Airstar: after 10 polls (~50 seconds) |
| 2 | Contact bank to confirm | Verify the final bank-side status of this remittance | — |
| 3a | Bank confirms success | Execute SetManualTransferDone in CRM, release Freeze + formally deduct | — |
| 3b | Bank confirms failure | Execute SetManualTransferFailed in CRM, release Freeze + rollback funds | — |
| 3c | Bank is also unsure | Maintain Freeze, continue following up. Never release the Freeze prematurely | — |
Freeze Exceptions Are Critical Alerts
If a Withdrawal fails but the Freeze is not properly released (system exception), the user's funds will be "stuck" — the balance shows as decreased but the Withdrawal didn't succeed either. This is a P0 critical alert item (Monitoring & Alerting System), requiring Operations to manually release the Freeze using CashWithdrawForceProcess.
Steps:
- Locate the Procedure in CRM
- Confirm bank-side status (not transferred / already transferred / uncertain)
- If bank has not transferred:
CashWithdrawForceProcess→ select "Rollback" → release Freeze + restore balance - If bank has already transferred:
CashWithdrawForceProcess→ select "Complete" → release Freeze + formally deduct - If bank is uncertain: do not take action, continue following up with the bank
Freeze type codes → Withdrawal Rules Handbook § Freeze Types
Procedure Lookup and Troubleshooting Guide
When a user reports "Deposit didn't arrive" or "Withdrawal is stuck in processing," Procedure is the core of issue resolution. Below are the investigation paths for Operations in CRM:
Deposit Issue Investigation
Withdrawal Issue Investigation
| Step | Check Item | Where to Check | Normal Value | Exception Handling |
|---|---|---|---|---|
| 1 | Withdrawal Task status | CRM → Withdrawal Management | DONE / PROCESSING | PENDING = Approval not complete |
| 2 | SBA Procedure status | CRM → SBA Management | end_ok | Locate based on current status (see status table above) |
| 3 | Procedure ext_status | Procedure details | transfer_done | Pinpoint exactly which step it's stuck at |
| 4 | Flow audit records | Procedure → View History | Statuses progressing normally over time | Find the last Flow entry to identify the bottleneck |
| 5 | reason field | Procedure details | Empty (when successful) | Non-empty = check the specific rejection/failure reason |
| 6 | transfer_ref_id | Procedure details | Bank reference number | Key identifier for bank reconciliation |
Key Search Fields
| Search Method | Field | Use Case |
|---|---|---|
| By user | nn_uid (NiuNiu ID) | View all Deposit/Withdrawal Procedures for a specific user |
| By Procedure | id | Directly locate by known Procedure ID |
| By bank reference | transfer_ref_id / transaction_id | Used during bank reconciliation |
| By time | created_at + status | Batch investigation of exception Procedures in a time range |
| By Channel | transfer_method | Investigate overall issues with a specific Channel |
Common Exception Scenarios
Deposit Exceptions
| # | Scenario | Symptoms | Possible Causes | Operations Handling |
|---|---|---|---|---|
| 1 | Procedure stuck at new(waiting) | Deposit request created but not credited for a long time | SBA service exception; triggered manual review rule but not flagged | Check SBA Deposit Orchestration service logs; view Flow records to confirm whether any status changes occurred |
Scenario 1 Suggested Response
"Your Deposit is being processed. The system is performing funds verification, which usually completes within a few minutes. If you still have not received the funds after 30 minutes, please provide your NiuNiu ID and we will prioritize the follow-up."
| 2 | Risk Control change failure | Procedure goes from pending → end_reject | Risk Control system rejection (internal rule triggered); asset change API exception | Check the reason field, contact the Risk Control team to confirm the cause | | 3 | Duplicate Deposit | Multiple Procedures created for the same Bank Statement | Deduplication mechanism failed (network timeout retry); bank pushed duplicate Bank Statements | Reverse the excess Deposits (CashDepositReverse), keeping the correct one | | 4 | Bank refund (BST) | Airstar Bank returns REFUNDED status | Bank-side Risk Control interception; insufficient balance in user's bank account | Confirm that credited funds have been reversed; notify user of the reason | | 5 | Deposit amount mismatch | Bank Statement amount doesn't match application amount | User manually changed the amount during transfer; cross-bank fees deducted | Route to manual Matching, confirm the actual credited amount | | 6 | BST Deposit polling timeout | Airstar Deposit with no result after 60 polls | Bank system delay; network exception | Check retry_count in AsbBstCreateResultJob; contact Airstar Bank to verify |
Withdrawal Exceptions
| # | Scenario | Symptoms | Possible Causes | Operations Handling |
|---|---|---|---|---|
| 1 | Freeze failure | Procedure goes from new(freeze) → pending(transfer_reject) | Insufficient available balance; Risk Control Freeze API exception | Check user's available balance; if balance is sufficient but still fails, contact Risk Control team |
| 2 | Deduction failure | Procedure goes from new(blank) → pending(transfer_reject) | Risk Control asset change API timeout; Freeze released but deduction not completed | Check if Risk Control system is healthy; view Flow records to pinpoint the deduction failure point |
| 3 | Bank rejects transfer | Procedure enters pending(return) | Incorrect Bank Card details (card number/SWIFT); bank-side Risk Control interception; receiving account exception | Check the bank error code; confirm Bank Card details are correct; funds automatically restored |
| 4 | Freeze not released | Withdrawal fails but user balance not restored | Unfreeze request failed to send; Risk Control system did not properly handle the release | Critical alert — use CashWithdrawForceProcess to force-release |
| 5 | Polling timeout (Airstar) | Procedure stuck at pending(transfer_auto) | Bank system delay; network exception | Contact Airstar Bank to confirm bank-side status, then manually set success or failure accordingly |
| 6 | Cross-day delay | BST Withdrawal pending(move_next) | Submitted outside trading hours (normal behavior) | Explain trading day rules to the user. Withdrawals outside trading hours are deferred to the next trading day |
Scenario 6 Suggested Response
"Your Withdrawal has been accepted. Since BST Channels process during trading hours, your Withdrawal will be automatically processed on the next trading day (Monday), with an estimated arrival in 1–2 business days. Thank you for your patience."
| 7 | Manual transfer not confirmed | Procedure stuck at pending(transfer_manual) for a long time | Operations has not completed the transfer in online banking; transfer completed but confirmation was forgotten | Confirm whether the transfer has been completed; if so, click "Confirm Transfer Complete" |
Scenario 7 Suggested Response
"Your Withdrawal is being processed. Our Operations team is completing the bank transfer. It is expected to be completed today; the arrival time depends on the receiving bank's processing speed (typically 1–3 business days)."
| 8 | Restoration failure | pending(return) stuck without transitioning to terminal state for a long time | Risk Control restoration API exception | Critical alert — user's money has "disappeared." Contact the technical team to investigate the restoration failure |
Next Steps
| I want to... | Go to |
|---|---|
| Learn about detailed BST bank comparisons | BST Overview |
| See the overall architecture and data flow | System Architecture & Data Flow |
| Look up SBA Procedure status codes | Withdrawal Data Dictionary |
| See Withdrawal Freeze and limit rules | Withdrawal Rules Handbook |
| Check Operations guides | Manual Matching Guide, Withdrawal Approval Guide |