With the recent advancements of sensory technologies (such as Kinect), perceiving reliably basic human actions have become tenable. If robots were to learn or interact with humans in a meaningful manner, the next foreseeable challenge to face robotic research in this area is toward the semantic understanding of human activities - enabling them to extract and determine higher level understanding.

In this research, we present a new methodology that account for the extraction of observed human behaviors with an estimation of the intended activities, follow by the automatic generation of action rules for the synthesis of robot behaviors.

Furthermore, we enhance the semantic representation with our reasoning system. It is important to mention that the obtained rules are preserved even when different kinds of kitchen scenarios are observed. In order to test the robustness of our results, we used three different kitchen activities:

• making a pancake,
• making a sandwich, and
• setting the table.

Moving beyond the state-of-the-art in imitation learning, ontology of behavioral rules from human observations can provide more powerful tools for the robots to learn from humans.