关键词: 微流控生物芯片, 完全可编程阀门阵列, 组件布局, 深度强化学习, 双重深度Q网络

Abstract: As a novel continuous-flow microfluidic biochip, the fully programmable valve array (FPVA) biochip boasts high flexibility and programmability. As an experimental platform, it  offers enhanced manipulation flexibility and enables personalized experimental workflow configurations. However, with advancements in chip manufacturing processes, the integration level of FPVA biochips has reached a high level. Combined with its high degree of freedom, this increases the difficulty in programming and designing FPVA biochips. Module placement is a critical step in biochip design. Previous studies typically employ heuristic algorithms for placement, which often yield limited results for discrete problems and pose challenges in parameter settings. Designing an efficient, user-friendly algorithms more suitable for discretized module placement can enhance the overall efficiency of the chip design process. Deep reinforcement learning (DRL) offers advantages in efficiency, adaptability, and flexibility. Agents, through continuous interaction with the environment, self-train and adjust, swiftly adapting to various complex variations and requirements to find optimal or near-optimal strategies. Compared to heuristic algorithms, DRL can better adapt to the environment and find global optimal placement solution. Therefore, this paper proposes a DRL-based module placement algorithm for FPVA biochips. It constructs an interactive environment for DRL agents within FPVA chips and employs the double deep Q-network to build a module placement decision model. Leveraging the rapid iteration capability of agents, it efficiently completes large-scale integrated module placement tasks under FPVA biochips. Moreover, by designing concurrent relationship constraints and placement area constraints to determine the concurrency between modules and restrict the placement area on the chip, the placement scheme can better conform to real-world scenarios, ensuring the correctness and feasibility of the placement scheme. Comparative experiments with state-of-the-art algorithm across multiple test cases demonstrate that the proposed algorithm can generate module placement schemes with shorter pre-routing wirelength and fewer unit reuse instances, thus providing  a high-quality placement scheme for subsequent routing stages.

Key words: microfluidic biochip, fully programmable valve array(FPVA), module placement, deep reinforcement learning(DRL), double deep Q-network(DDQN)