Content-Disposition: inline
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset="US-ASCII"
MIME-Version: 1.0
Date: Sun, 22 Nov 2020 20:17:39 +0000
From: "Jared D. McNeill" <jmcneill@netbsd.org>
Subject: CVS commit: src/sys/arch/arm/cortex
To: source-changes@NetBSD.org
Message-Id: <20201122201739.B719CFA9D@cvs.NetBSD.org>
Sender: source-changes-owner@NetBSD.org
Precedence: bulk
Reply-To: source-changes-d@NetBSD.org
Mail-Reply-To: "Jared D. McNeill" <jmcneill@netbsd.org>
Mail-Followup-To: source-changes-d@NetBSD.org

Module Name:	src
Committed By:	jmcneill
Date:		Sun Nov 22 20:17:39 UTC 2020

Modified Files:
	src/sys/arch/arm/cortex: gicv3.c

Log Message:
Fix interrupt priorities on N1 SDP.

The GICv3 architecture specification is not clear on the NS view of
priority registers, and there doesn't seem to be any consistency in how
these are implemented in both real and emulated environments.

The previous fix for this issue was meant to detect what we thought at the
time was a bug on the Rockchip RK3399. At that time the theory was somehow
EL1 has a secure view of the hardware, and this is causing us to have the
wrong view of IPRIORITYRn based on IHI0069F section 4.8.6 "Software
accesses of interrupt priority". But it turns out that this is not the
full picture. While I was able to confirm that yes, we do have secure
access to the GIC on RK3399 from EL1, the view of IPRIORITYRn differs
depending on whether you are using the Rockchip TF-A as included with
https://github.com/ayufan-rock64/linux-u-boot (shifted view), or mainline
TF-A from pkgsrc (unshifted view).

So to detect this quirk, we need three things: A method to detect if we
have S access to GIC registers, a method to see how many PMR bits are
implemented, and a method to see how many IPRIORITYRn bits are implemented.

To detect S access, we can try to toggle GICD_CTRL.EnableGrp1S. This bit
is either RES0 (security extensions not implemented), RAZ/WI (non-secure
access in two security state systems) or RW (secure access in two security
state systems).

To read the number of PMR and IPRIORITYRn bits supported, we can write all
1s to the register fields and read them back.

For the RK3399 (Rockchip TF-A) quirk, we assume a shifted view of
IPRIORITYRn if we have detected S accesses, and the PMR and IPRIORITYRn
values differ. The S access test is required because some real hardware
implementations (Ampere eMAG) were observed to report different PMR and
IPRIORITYRn masks, but present an unshifted view of IPRIORITYRn.

During testing, I also discovered that QEMU 5.1 requires this shifted view
workaround as well -- as far as I can tell, this is a QEMU bug. We can't
detect it the same way as RK3399 because security is disabled in the
emulated GIC, and the PMR and IPRIORITYRn tests both return 0xff! So now
if the GICv3 driver sees this configuration, it assumes that the shifted
view is required.

Honestly, this feature is so poorly documented that maybe it is better to
give up on HW priorities and preemption and use a single flat model like
Linux and FreeBSD does.

Tested on Arm N1 SDP, ROCKpro64 (RK3399) with Rockchip and pkgsrc TF-A,
Pinebook Pro (RK3399), Lenovo HR330A (Ampere eMAG), QEMU 5.1 (gic-version=3),
AWS EC2 a1.medium (Graviton).


To generate a diff of this commit:
cvs rdiff -u -r1.33 -r1.34 src/sys/arch/arm/cortex/gicv3.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.