Messages in this thread | | | From | Ley Foon Tan <> | Date | Thu, 7 Oct 2021 18:10:26 +0800 | Subject | Re: [RFC PATCH v8 09/21] riscv: Add task switch support for vector |
| |
On Tue, Oct 5, 2021 at 11:47 PM Greentime Hu <greentime.hu@sifive.com> wrote: > > Ley Foon Tan <lftan.linux@gmail.com> 於 2021年10月5日 週二 上午10:12寫道: > > > > On Mon, Oct 4, 2021 at 8:41 PM Greentime Hu <greentime.hu@sifive.com> wrote: > > > > > > Ley Foon Tan <lftan.linux@gmail.com> 於 2021年10月1日 週五 上午10:46寫道: > > > > > > > > On Wed, Sep 29, 2021 at 11:54 PM Darius Rad <darius@bluespec.com> wrote: > > > > > > > > > > On Tue, Sep 28, 2021 at 10:56:52PM +0800, Greentime Hu wrote: > > > > > > Darius Rad <darius@bluespec.com> 於 2021年9月13日 週一 下午8:21寫道: > > > > > > > > > [....] > > > > > > > > > > > > > > > > > > So this will unconditionally enable vector instructions, and allocate > > > > > > > memory for vector state, for all processes, regardless of whether vector > > > > > > > instructions are used? > > > > > > > > > > > > > > > > > > > Hi Darius, > > > > > > > > > > > > Yes, it will enable vector if has_vector() is true. The reason that we > > > > > > choose to enable and allocate memory for user space program is because > > > > > > we also implement some common functions in the glibc such as memcpy > > > > > > vector version and it is called very often by every process. So that > > > > > > we assume if the user program is running in a CPU with vector ISA > > > > > > would like to use vector by default. If we disable it by default and > > > > > > make it trigger the illegal instruction, that might be a burden since > > > > > > almost every process will use vector glibc memcpy or something like > > > > > > that. > > > > > > > > > > Do you have any evidence to support the assertion that almost every process > > > > > would use vector operations? One could easily argue that the converse is > > > > > true: no existing software uses the vector extension now, so most likely a > > > > > process will not be using it. > > > > > > > > > > > > > > > > > > Given the size of the vector state and potential power and performance > > > > > > > implications of enabling the vector engine, it seems like this should > > > > > > > treated similarly to Intel AMX on x86. The full discussion of that is > > > > > > > here: > > > > > > > > > > > > > > https://lore.kernel.org/lkml/CALCETrW2QHa2TLvnUuVxAAheqcbSZ-5_WRXtDSAGcbG8N+gtdQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org/ > > > > > > > > > > > > > > The cover letter for recent Intel AMX patches has a summary of the x86 > > > > > > > implementation: > > > > > > > > > > > > > > https://lore.kernel.org/lkml/20210825155413.19673-1-chang.seok.bae@intel.com/ > > > > > > > > > > > > > > If RISC-V were to adopt a similar approach, I think the significant > > > > > > > points are: > > > > > > > > > > > > > > 1. A process (or thread) must specifically request the desire to use > > > > > > > vector extensions (perhaps with some new arch_prctl() API), > > > > > > > > > > > > > > 2. The kernel is free to deny permission, perhaps based on > > > > > > > administrative rules or for other reasons, and > > > > > > > > > > > > > > 3. If a process attempts to use vector extensions before doing the > > > > > > > above, the process will die due to an illegal instruction. > > > > > > > > > > > > Thank you for sharing this, but I am not sure if we should treat > > > > > > vector like AMX on x86. IMHO, compiler might generate code with vector > > > > > > instructions automatically someday, maybe we should treat vector > > > > > > extensions like other extensions. > > > > > > If user knows the vector extension is supported in this CPU and he > > > > > > would like to use it, it seems we should let user use it directly just > > > > > > like other extensions. > > > > > > If user don't know it exists or not, user should use the library API > > > > > > transparently and let glibc or other library deal with it. The glibc > > > > > > ifunc feature or multi-lib should be able to choose the correct > > > > > > implementation. > > > > > > > > > > What makes me think that the vector extension should be treated like AMX is > > > > > that they both (1) have a significant amount of architectural state, and > > > > > (2) likely have a significant power and/or area impact on (non-emulated) > > > > > designs. > > > > > > > > > > For example, I think it is possible, maybe even likely, that vector > > > > > implementations will have one or more of the following behaviors: > > > > > > > > > > 1. A single vector unit shared among two or more harts, > > > > > > > > > > 2. Additional power consumption when the vector unit is enabled and idle > > > > > versus not being enabled at all, > > > > > > > > > > 3. For a system which supports variable operating frequency, a reduction > > > > > in the maximum frequency when the vector unit is enabled, and/or > > > > > > > > > > 4. The inability to enter low power states and/or delays to low power > > > > > states transitions when the vector unit is enabled. > > > > > > > > > > None of the above constraints apply to more ordinary extensions like > > > > > compressed or the various bit manipulation extensions. > > > > > > > > > > The discussion I linked to has some well reasoned arguments on why > > > > > substantial extensions should have a mechanism to request using them by > > > > > user space. The discussion was in the context of Intel AMX, but applies to > > > > > further x86 extensions, and I think should also apply to similar extensions > > > > > on RISC-V, like vector here. > > > > > > > > > There is possible use case where not all cores support vector > > > > extension due to size, area and power. > > > > Perhaps can have the mechanism or flow to determine the > > > > application/thread require vector extension or it specifically request > > > > the desire to use > > > > vector extensions. Then this app/thread run on cpu with vector > > > > extension capability only. > > > > > > > > > > IIRC, we assume all harts has the same ability in Linux because of SMP > > > assumption. > > > If we have more information of hw capability and we may use this > > > information for scheduler to switch the correct process to the correct > > > CPU. > > > Do you have any idea how to implement it in Linux kernel? Maybe we can > > > list in the TODO list. > > I think we can refer to other arch implementations as reference: > > > > 1. ARM64 supports 32-bit thread on asymmetric AArch32 systems. There > > is a flag in ELF to check, then start the thread on the core that > > supports 32-bit execution. This patchset is merged to mainline 5.15. > > https://lore.kernel.org/linux-arm-kernel/20210730112443.23245-8-will@kernel.org/T/ > > Wow! This is useful for AMP. > > > > > 2. Link shared by Darius, on-demand request implementation on Intel AMX > > https://lore.kernel.org/lkml/20210825155413.19673-1-chang.seok.bae@intel.com/ > > > > glibc support optimized library functions with vector, this is enabled > > by default if compiler is with vector extension enabled? If yes, then > > most of the app required vector core. > > As I mentioned earlier, glibc ifunc will solve this issue. The > Linux/glibc can run on platform with vector or without vector and > glibc will use the information get from Linux kernel and using ifunc > to decide whether it should use the vector version or not. > Which means even your toolchain has vector glibc support and your > Linux kernel told the glibc this platform doesn't support vector then > the ifunc mechanism will choose the non-vector version ones. Okay. Then Linux kernel needs to report vector capability as per core feature, if not all SMP cores support vector.
| |