Abstract:

One common approach of selfadaptive middleware is to incorporate a control loop that monitors, analyzes, decides and executes over a target system with predefined strategies. Such approach is an offline adaptation where strategies or adaptive models are statically determined so as not to change with environment. Aiming at the problem, an online updating mechanism of selfadaptive middleware based on reinforcement learning is proposed to solve the problems of conflict resolution and realtime system effectiveness evaluation, and an online updating method of selfadaptive policy based on reinforcement learning is designed, thus enhancing intelligence, flexibility and reaction capability. Finally, the corresponding system OUSAM is implemented and the effectiveness and feasibility of the mechanism is validated on OUSAM.

Key words: self-adaptive middleware;online updating;intelligent decision;reinforcement learning