Title: Supporting Real-time guest OS with Xen-ARM Mobile Virtual Machine
‘Draws attention’
Virtualization in mobile systems presents many advantages regarding reliability, security, and flexible user customizability, etc.
‘Unique characteristics of mobile virtualization’
Mobile virtualization has different requirements such as real-time support, small footprint, performance with limited hardware resources. In this work, we focus on supporting heterogeneous guest OSs support, which is a unique requirement in mobile systems virtualization. Those mobile devices requires real-time performance that only RTOS can guarantee as well as rich functionalities that GPOS is able to provide.
‘Introduce difficulties - Porting is nothing more than executing’
In general, it is difficult to support heterogeneous guest OSs over the same physical machine. It is definitely not sufficient for porting two guest OSs, and running within the single physical machine. That is because heterogeneous guest OSs presents different policies, that might conflict each other, and scheduling is one of most significant issues to be resolved. A real-time guest OS has its unique scheduling policy and task model so that all the tasks can meet their given deadlines. On the other hand, fairness and reasonable response time are two primary concerns for a general-purpose guest OS. Those scheduling policies can conflict. For example, a RT guest OS requires immediate task scheduling in order to meet the deadline of a task, that would break the fairness of task scheduling of the GP guest OS, or vice versa. Namely, the difficulty rises when two guest OSs contend for CPU with different conflicting policies.
‘It is the hypervisor that has to arbitrate scheduling in such an environment’
Hierarchical scheduling is an approach that most hypervisors use. In a hierarchical scheduling system, the hypervisor scheduler strictly partition the CPU utilization, and gives the CPU bandwidth to each guest OS. Since each guest OS runs with explicitly given CPU bandwidth, the scheduling policy of a guest OS doesn’t break the scheduling policy of another guest OS.
‘Difficulties are two-fold; what’s more needed for real-time guest OSs’
On one hand, hierarchical scheduling implements resource partition scheduling that strictly isolates performance among guest OSs. However, it is not sufficient to support a real-time guest OS since the requirements are multi-disciplinary rather than simple CPU bandwidth allocation. To address real-time support in a virtualization system, I/O latency issue has to be addressed as well as performance isolation. Since a real-time guest OS has requirements not only for periodic CPU-bound tasks, but also for unpredictable and aperiodic I/O tasks, static CPU bandwidth allocation is not sufficient to deal with I/O latency issues even though hierarchical scheduling is used.
‘KU-approach for real-time virtualization’
1. ‘PARFAIT is an effort to schedule heterogeneous guest OSs over a single physical machine’
Parfait is a scheduling framework that incorporates both real-time guest OSs and non-real-time guest OSs, at the same time. It allocates static amount of CPU bandwidth to real-time guest OSs, at first hand. Secondly, it fairly distributes the CPU bandwidth among the other GP guest OSs. It enables to isolate performance among RTOS and GPOS so that one of guest OSs doesn’t have negative performance impact to another. To incorporate deterministic scheduling theory into a practical system, we performed quantization analysis, and presents an algorithm that provides a scheduling parameter that guarantees schedulability of a virtual machine.
2. ‘To guarantee I/O latency, scheduling optimization has provided’
To guarantee I/O latency, scheduling algorithm has to be carefully optimized. We particularly focus on the Xen-ARM’s split driver in their paravirtualization architecture. It functionally isolates device drivers from user domains so that the potentially faulty device driver cannot compromise the entire system. Although it enhances the reliability of the entire system, it causes additional latency by frequent switching back and forth to the driver domain. With the original credit scheduling policy in Xen-ARM, it presents serious performance degradation. Increased I/O latency might lead to call drops/misses, which can be regarded a serious system defect in mobile systems. To deal with I/O latency, we prioritize the driver domain so that it cannot be scheduled out by another domain. On top of that, we take advantage of hardware specific architectural state. In addition to the hypervisor scheduler, we optimized the guest OS so that the driver domain’s physical interrupt handling cannot be discontinued by virtualized interrupt disable/enable routine.
Tuesday, June 28, 2011
Monday, June 20, 2011
Minimizing I/O latency in Xen-ARM virtual machine
Title: Minimizing I/O latency in Xen-ARM virtual machine
Recently, virtualization takes attention to mobile systems because it enhances reliability, security, and flexible user customizability. In recent mobile systems, security and user customizability are more important than ever before since it manages all private personal information and more users want to customize the device as their own way. In addition, those mobile systems recently serve as a business assistant, so reliability and security became serious concerns. Mobile systems virtualization can address those issues in an efficient manner. For example, Xen’s split driver presents enhanced reliability by isolating a faulty driver domain or virus-infected malicious domain from trusted user domains.
There are several hypervisors for mobile systems. OKL4 microvisor is one of the most representative hypervisors. OKL4 microvisor serves as trusted computing base, by using formally verified kernel, and it presents efficient virtualization enough to be used in commercial mobile devices. VirtualLogix-VLX is another commercial hypervisor for mobile systems. It supports shared driver model that enables both real-time guest OS and non-real-time guest OS to share a physical device. It also presents comparable performance to native systems. Xen-ARM is yet another approach for incorporating Xen into ARM-based mobile systems. It presents small footprint, secure access control.
Despite the advantages of virtualization in mobile systems, performance issue has to be addressed at the same time. Since mobile systems have limited hardware resources (e.g. power supply, memory size, CPU budgets, etc.,) the implementation has to be efficient enough to overcome the performance overhead from virtualization. Among various performance problems, I/O latency is one of serious concerns because many mobile systems have communication device that needs to handle signaling protocols in a timely manner. In particular, the split driver model in Xen-ARM presents additional latency larger than 1ms, which could disconnect the mobile device from the networks.
In this paper, we analyze the two major sources of I/O latency in Xen-ARM’s split driver model, and propose a new scheduler in order for a real-time guest OS to handle I/O in a timely manner. Our analysis, at first, reveals that interrupt delivery to driver domain can be delayed because interrupt handling is virtualized when running within a guest OS. Although a driver domain disables virtual interrupts, physical interrupt can still be occurred, which might trigger inter-VM scheduling. Thus, physical driver in the driver domain cannot guaranteed to be scheduled in a timely manner. Secondly, additional latency is observed by inter-VM scheduling between the execution of backend and frontend drivers. Even though the existing scheduler supports I/O boost priority, I/O boost can be easily negated by multiple boost or ignored by fair scheduling policy.
To reduce latency while retaining the split driver model, we improve the scheduler so that the driver domain cannot be scheduled out when it disables virtual interrupts and the scheduler provide static priorities to guest OSs in order to provide a time bounded I/O latency. With our scheduler, all I/O interrupt at real-time guest OS is handled within 1ms while retaining split driver model, which is more strict latency bound than native ARM-Linux.
This paper consists of five sections. Section 2 introduces related work on embedded hypervisors and
Related Work
OKL4 microvisor is one of representative hypervisors for mobile systems. It is originated from L4 microkernel that overcomes the performance problems of first generation of microkernels. With extensive architectural optimization, L4Linux achieves good performance as close as native Linux, and OKL4 microvisor presents flexible architecture to incorporate multiple commodity operating systems. On top of that, it supports user-level device driver, which de-privileges driver domain in order to protect trusted user domains from the faulty driver implementation. In addition to performance, it is verified with formal language, so they claim that their hypervisor is bug-free, and thereby it is trustworthy.
Regarding the real-time performance, L4/Fiasco has been developed for real-time and embedded systems. It supports real-time OSs with several optimizations including direct task switching, time slice donation, and user-level reflective scheduling, etc. Although L4’s effort on real-time support presents impressive results in both desktop systems and embedded systems, their performance impact has not been thoroughly analyzed with more complex systems with virtualization such as multiple guest OS environment or multiple driver interactions.
Another hypervisor, VirtualLogix-VLX is a mobile hypervisor for incorporating multiple guest OSs over a single physical hardware platform. VLX supports shared device model that enables to share devices among multiple guest OSs similar to Xen. In their shared driver model, one guest OS has the device driver, and it is shared by communication channel provided by the hypervisor. VLX optimizes performance by locating the bridge driver inside the hypervisor in order to mitigate demultiplexing overhead. However, as shown in our study, using a real-time guest OS along with the driver domain presents degraded performance under some circumtances in terms of both response time and fair CPU utilization.
Xen-ARM is another hypervisor for ARM-based mobile systems. It is an ARM-port for Xen hypervisor, and it presents small footprint and secure access control. Xen-ARM supports the split driver model that has presented in Xen paravirtualization implementation. Xen’s split driver model enhances the reliability by isolating potentially faulty and untrusted device drivers from the trusted user domain. It separately locates IDD (Isolated Driver Domain), that has the physical drivers for virtualized devices, from user domains. IDD performs all physical I/O operations on behalf of the guest OSs, and hypervisor arbitrates only requests and responses among them. Since all drivers are excluded from the hypervisor, the hypervisor itself claims that it is more reliable. To mitigate the performance overhead from synchronous inter-VM communication, Xen-ARM extensively uses asynchronous I/O between IDD and user domain.
Xen-ARM supports the credit scheduler that is also used in Xen. It implements weighted round-robin scheduling algorithm and work-conserving mode. The scheduler periodically distributes the same amount of credit to all VCPUs, and debits as much as it executes. At the beginning, VCPU has UNDER priority, and the priority is changed to OVER when the VCPU consumes all its credits, and has minus credit values. The scheduler selects a VCPU among ones that has UNDER priority. To mitigate the worst-case response time for I/O tasks, it supports BOOST priority that temporarily prioritize interrupt-pending domain. BOOST is the highest priority among UNDER, OVER and BOOST, and the VCPU immediately preempt the running VCPU so that the I/O handling domain can be scheduled in a timely manner. However, it is beneficial only in limited cases because I/O boost in the credit scheduler doesn’t consider the urgency among boosted domains. In addition, boost is not always applied since the scheduler primarily focuses on fairness among CPU-bound domains. Namely, the credit scheduler is not feasible to support time-sensitive virtual machines.
The lack of time-sensitive applications support leads further studies on the credit scheduler. Task-aware scheduler proposed by Hwanjoo et. al. observes task scheduling within the guest OSs, and determines the characteristics of the guest OS whether it is CPU-bound or I/O-bound. Then, based on the estimation, the scheduler adaptively applies boost policy (named partial boost) so that I/O-bound domain can get guaranteed response time. Lee min et. al. proposed a new scheduler improving soft real-time performance. The scheduler estimates average execution time of the VCPU, and calculates the scheduling priority based on the deadline estimation of the VCPU. The scheduler further addresses multicore issues regarding task affinity to maximize cache utilization.
Yanyan et. al. presented I/O scheduling within multicore virtual machine. Proposed scheduler differently allocates CPU cores by workload characteristics: driver core, fast-tick core and slow-tick core. Driver core is dedicated for driver domain, fast tick cores are for I/O-bound domains, and slow-tick cores are for CPU-bound domains. Driver core is dedicated to the driver domain in order not to be preempted by another domain. Driver core minimizes the scheduling latency for physical driver execution and respective backend driver operations. Fast tick core reduces response time because the scheduler is invoked frequently according to the reduced tick interval. On the other hand, CPU-bound domains are running on the slow-tick cores in order to maximize cache utilization and throughput. Their approach is plausible, but in modern servers already have fast ticks with considerably small overhead, such as HRTimers in Linux. In addition, in embedded systems, high frequency timers are not applicable in many cases because the performance overhead for small tick interval is much significant than servers or desktop systems.
Latency presented in Xen-Arm
Recently, virtualization takes attention to mobile systems because it enhances reliability, security, and flexible user customizability. In recent mobile systems, security and user customizability are more important than ever before since it manages all private personal information and more users want to customize the device as their own way. In addition, those mobile systems recently serve as a business assistant, so reliability and security became serious concerns. Mobile systems virtualization can address those issues in an efficient manner. For example, Xen’s split driver presents enhanced reliability by isolating a faulty driver domain or virus-infected malicious domain from trusted user domains.
There are several hypervisors for mobile systems. OKL4 microvisor is one of the most representative hypervisors. OKL4 microvisor serves as trusted computing base, by using formally verified kernel, and it presents efficient virtualization enough to be used in commercial mobile devices. VirtualLogix-VLX is another commercial hypervisor for mobile systems. It supports shared driver model that enables both real-time guest OS and non-real-time guest OS to share a physical device. It also presents comparable performance to native systems. Xen-ARM is yet another approach for incorporating Xen into ARM-based mobile systems. It presents small footprint, secure access control.
Despite the advantages of virtualization in mobile systems, performance issue has to be addressed at the same time. Since mobile systems have limited hardware resources (e.g. power supply, memory size, CPU budgets, etc.,) the implementation has to be efficient enough to overcome the performance overhead from virtualization. Among various performance problems, I/O latency is one of serious concerns because many mobile systems have communication device that needs to handle signaling protocols in a timely manner. In particular, the split driver model in Xen-ARM presents additional latency larger than 1ms, which could disconnect the mobile device from the networks.
In this paper, we analyze the two major sources of I/O latency in Xen-ARM’s split driver model, and propose a new scheduler in order for a real-time guest OS to handle I/O in a timely manner. Our analysis, at first, reveals that interrupt delivery to driver domain can be delayed because interrupt handling is virtualized when running within a guest OS. Although a driver domain disables virtual interrupts, physical interrupt can still be occurred, which might trigger inter-VM scheduling. Thus, physical driver in the driver domain cannot guaranteed to be scheduled in a timely manner. Secondly, additional latency is observed by inter-VM scheduling between the execution of backend and frontend drivers. Even though the existing scheduler supports I/O boost priority, I/O boost can be easily negated by multiple boost or ignored by fair scheduling policy.
To reduce latency while retaining the split driver model, we improve the scheduler so that the driver domain cannot be scheduled out when it disables virtual interrupts and the scheduler provide static priorities to guest OSs in order to provide a time bounded I/O latency. With our scheduler, all I/O interrupt at real-time guest OS is handled within 1ms while retaining split driver model, which is more strict latency bound than native ARM-Linux.
This paper consists of five sections. Section 2 introduces related work on embedded hypervisors and
Related Work
OKL4 microvisor is one of representative hypervisors for mobile systems. It is originated from L4 microkernel that overcomes the performance problems of first generation of microkernels. With extensive architectural optimization, L4Linux achieves good performance as close as native Linux, and OKL4 microvisor presents flexible architecture to incorporate multiple commodity operating systems. On top of that, it supports user-level device driver, which de-privileges driver domain in order to protect trusted user domains from the faulty driver implementation. In addition to performance, it is verified with formal language, so they claim that their hypervisor is bug-free, and thereby it is trustworthy.
Regarding the real-time performance, L4/Fiasco has been developed for real-time and embedded systems. It supports real-time OSs with several optimizations including direct task switching, time slice donation, and user-level reflective scheduling, etc. Although L4’s effort on real-time support presents impressive results in both desktop systems and embedded systems, their performance impact has not been thoroughly analyzed with more complex systems with virtualization such as multiple guest OS environment or multiple driver interactions.
Another hypervisor, VirtualLogix-VLX is a mobile hypervisor for incorporating multiple guest OSs over a single physical hardware platform. VLX supports shared device model that enables to share devices among multiple guest OSs similar to Xen. In their shared driver model, one guest OS has the device driver, and it is shared by communication channel provided by the hypervisor. VLX optimizes performance by locating the bridge driver inside the hypervisor in order to mitigate demultiplexing overhead. However, as shown in our study, using a real-time guest OS along with the driver domain presents degraded performance under some circumtances in terms of both response time and fair CPU utilization.
Xen-ARM is another hypervisor for ARM-based mobile systems. It is an ARM-port for Xen hypervisor, and it presents small footprint and secure access control. Xen-ARM supports the split driver model that has presented in Xen paravirtualization implementation. Xen’s split driver model enhances the reliability by isolating potentially faulty and untrusted device drivers from the trusted user domain. It separately locates IDD (Isolated Driver Domain), that has the physical drivers for virtualized devices, from user domains. IDD performs all physical I/O operations on behalf of the guest OSs, and hypervisor arbitrates only requests and responses among them. Since all drivers are excluded from the hypervisor, the hypervisor itself claims that it is more reliable. To mitigate the performance overhead from synchronous inter-VM communication, Xen-ARM extensively uses asynchronous I/O between IDD and user domain.
Xen-ARM supports the credit scheduler that is also used in Xen. It implements weighted round-robin scheduling algorithm and work-conserving mode. The scheduler periodically distributes the same amount of credit to all VCPUs, and debits as much as it executes. At the beginning, VCPU has UNDER priority, and the priority is changed to OVER when the VCPU consumes all its credits, and has minus credit values. The scheduler selects a VCPU among ones that has UNDER priority. To mitigate the worst-case response time for I/O tasks, it supports BOOST priority that temporarily prioritize interrupt-pending domain. BOOST is the highest priority among UNDER, OVER and BOOST, and the VCPU immediately preempt the running VCPU so that the I/O handling domain can be scheduled in a timely manner. However, it is beneficial only in limited cases because I/O boost in the credit scheduler doesn’t consider the urgency among boosted domains. In addition, boost is not always applied since the scheduler primarily focuses on fairness among CPU-bound domains. Namely, the credit scheduler is not feasible to support time-sensitive virtual machines.
The lack of time-sensitive applications support leads further studies on the credit scheduler. Task-aware scheduler proposed by Hwanjoo et. al. observes task scheduling within the guest OSs, and determines the characteristics of the guest OS whether it is CPU-bound or I/O-bound. Then, based on the estimation, the scheduler adaptively applies boost policy (named partial boost) so that I/O-bound domain can get guaranteed response time. Lee min et. al. proposed a new scheduler improving soft real-time performance. The scheduler estimates average execution time of the VCPU, and calculates the scheduling priority based on the deadline estimation of the VCPU. The scheduler further addresses multicore issues regarding task affinity to maximize cache utilization.
Yanyan et. al. presented I/O scheduling within multicore virtual machine. Proposed scheduler differently allocates CPU cores by workload characteristics: driver core, fast-tick core and slow-tick core. Driver core is dedicated for driver domain, fast tick cores are for I/O-bound domains, and slow-tick cores are for CPU-bound domains. Driver core is dedicated to the driver domain in order not to be preempted by another domain. Driver core minimizes the scheduling latency for physical driver execution and respective backend driver operations. Fast tick core reduces response time because the scheduler is invoked frequently according to the reduced tick interval. On the other hand, CPU-bound domains are running on the slow-tick cores in order to maximize cache utilization and throughput. Their approach is plausible, but in modern servers already have fast ticks with considerably small overhead, such as HRTimers in Linux. In addition, in embedded systems, high frequency timers are not applicable in many cases because the performance overhead for small tick interval is much significant than servers or desktop systems.
Latency presented in Xen-Arm
Subscribe to:
Posts (Atom)