In allusion to the problem that Q-Learning

which was used discount reward as the evaluation criterion

could not show the affect of the action to the next situation

AR-Q-Learning was put forward based on the average reward and Q-Learning.In allusion to the curse of dimensionality

which meant that the computational requirement grew exponen-tially with the number of the state variable.Minimum state method was put forward.AR-Q-Learning and minimum state method were used in reinforcement learning for Blocks World

and the result of the experiment shows that the method has the characteristic of aftereffect and converges more faster than Q-Learning

and at the same time

solve the curse of di-mensionality in a certain extent in Blocks World.