61307b7be4
documented (hopefully adequately) in the respective changelogs. Notable
series include:
- Lucas Stach has provided some page-mapping
cleanup/consolidation/maintainability work in the series "mm/treewide:
Remove pXd_huge() API".
- In the series "Allow migrate on protnone reference with
MPOL_PREFERRED_MANY policy", Donet Tom has optimized mempolicy's
MPOL_PREFERRED_MANY mode, yielding almost doubled performance in one
test.
- In their series "Memory allocation profiling" Kent Overstreet and
Suren Baghdasaryan have contributed a means of determining (via
/proc/allocinfo) whereabouts in the kernel memory is being allocated:
number of calls and amount of memory.
- Matthew Wilcox has provided the series "Various significant MM
patches" which does a number of rather unrelated things, but in largely
similar code sites.
- In his series "mm: page_alloc: freelist migratetype hygiene" Johannes
Weiner has fixed the page allocator's handling of migratetype requests,
with resulting improvements in compaction efficiency.
- In the series "make the hugetlb migration strategy consistent" Baolin
Wang has fixed a hugetlb migration issue, which should improve hugetlb
allocation reliability.
- Liu Shixin has hit an I/O meltdown caused by readahead in a
memory-tight memcg. Addressed in the series "Fix I/O high when memory
almost met memcg limit".
- In the series "mm/filemap: optimize folio adding and splitting" Kairui
Song has optimized pagecache insertion, yielding ~10% performance
improvement in one test.
- Baoquan He has cleaned up and consolidated the early zone
initialization code in the series "mm/mm_init.c: refactor
free_area_init_core()".
- Baoquan has also redone some MM initializatio code in the series
"mm/init: minor clean up and improvement".
- MM helper cleanups from Christoph Hellwig in his series "remove
follow_pfn".
- More cleanups from Matthew Wilcox in the series "Various page->flags
cleanups".
- Vlastimil Babka has contributed maintainability improvements in the
series "memcg_kmem hooks refactoring".
- More folio conversions and cleanups in Matthew Wilcox's series
"Convert huge_zero_page to huge_zero_folio"
"khugepaged folio conversions"
"Remove page_idle and page_young wrappers"
"Use folio APIs in procfs"
"Clean up __folio_put()"
"Some cleanups for memory-failure"
"Remove page_mapping()"
"More folio compat code removal"
- David Hildenbrand chipped in with "fs/proc/task_mmu: convert hugetlb
functions to work on folis".
- Code consolidation and cleanup work related to GUP's handling of
hugetlbs in Peter Xu's series "mm/gup: Unify hugetlb, part 2".
- Rick Edgecombe has developed some fixes to stack guard gaps in the
series "Cover a guard gap corner case".
- Jinjiang Tu has fixed KSM's behaviour after a fork+exec in the series
"mm/ksm: fix ksm exec support for prctl".
- Baolin Wang has implemented NUMA balancing for multi-size THPs. This
is a simple first-cut implementation for now. The series is "support
multi-size THP numa balancing".
- Cleanups to vma handling helper functions from Matthew Wilcox in the
series "Unify vma_address and vma_pgoff_address".
- Some selftests maintenance work from Dev Jain in the series
"selftests/mm: mremap_test: Optimizations and style fixes".
- Improvements to the swapping of multi-size THPs from Ryan Roberts in
the series "Swap-out mTHP without splitting".
- Kefeng Wang has significantly optimized the handling of arm64's
permission page faults in the series
"arch/mm/fault: accelerate pagefault when badaccess"
"mm: remove arch's private VM_FAULT_BADMAP/BADACCESS"
- GUP cleanups from David Hildenbrand in "mm/gup: consistently call it
GUP-fast".
- hugetlb fault code cleanups from Vishal Moola in "Hugetlb fault path to
use struct vm_fault".
- selftests build fixes from John Hubbard in the series "Fix
selftests/mm build without requiring "make headers"".
- Memory tiering fixes/improvements from Ho-Ren (Jack) Chuang in the
series "Improved Memory Tier Creation for CPUless NUMA Nodes". Fixes
the initialization code so that migration between different memory types
works as intended.
- David Hildenbrand has improved follow_pte() and fixed an errant driver
in the series "mm: follow_pte() improvements and acrn follow_pte()
fixes".
- David also did some cleanup work on large folio mapcounts in his
series "mm: mapcount for large folios + page_mapcount() cleanups".
- Folio conversions in KSM in Alex Shi's series "transfer page to folio
in KSM".
- Barry Song has added some sysfs stats for monitoring multi-size THP's
in the series "mm: add per-order mTHP alloc and swpout counters".
- Some zswap cleanups from Yosry Ahmed in the series "zswap same-filled
and limit checking cleanups".
- Matthew Wilcox has been looking at buffer_head code and found the
documentation to be lacking. The series is "Improve buffer head
documentation".
- Multi-size THPs get more work, this time from Lance Yang. His series
"mm/madvise: enhance lazyfreeing with mTHP in madvise_free" optimizes
the freeing of these things.
- Kemeng Shi has added more userspace-visible writeback instrumentation
in the series "Improve visibility of writeback".
- Kemeng Shi then sent some maintenance work on top in the series "Fix
and cleanups to page-writeback".
- Matthew Wilcox reduces mmap_lock traffic in the anon vma code in the
series "Improve anon_vma scalability for anon VMAs". Intel's test bot
reported an improbable 3x improvement in one test.
- SeongJae Park adds some DAMON feature work in the series
"mm/damon: add a DAMOS filter type for page granularity access recheck"
"selftests/damon: add DAMOS quota goal test"
- Also some maintenance work in the series
"mm/damon/paddr: simplify page level access re-check for pageout"
"mm/damon: misc fixes and improvements"
- David Hildenbrand has disabled some known-to-fail selftests ni the
series "selftests: mm: cow: flag vmsplice() hugetlb tests as XFAIL".
- memcg metadata storage optimizations from Shakeel Butt in "memcg:
reduce memory consumption by memcg stats".
- DAX fixes and maintenance work from Vishal Verma in the series
"dax/bus.c: Fixups for dax-bus locking".
-----BEGIN PGP SIGNATURE-----
iHUEABYIAB0WIQTTMBEPP41GrTpTJgfdBJ7gKXxAjgUCZkgQYwAKCRDdBJ7gKXxA
jrdKAP9WVJdpEcXxpoub/vVE0UWGtffr8foifi9bCwrQrGh5mgEAx7Yf0+d/oBZB
nvA4E0DcPrUAFy144FNM0NTCb7u9vAw=
=V3R/
-----END PGP SIGNATURE-----
Merge tag 'mm-stable-2024-05-17-19-19' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull mm updates from Andrew Morton:
"The usual shower of singleton fixes and minor series all over MM,
documented (hopefully adequately) in the respective changelogs.
Notable series include:
- Lucas Stach has provided some page-mapping cleanup/consolidation/
maintainability work in the series "mm/treewide: Remove pXd_huge()
API".
- In the series "Allow migrate on protnone reference with
MPOL_PREFERRED_MANY policy", Donet Tom has optimized mempolicy's
MPOL_PREFERRED_MANY mode, yielding almost doubled performance in
one test.
- In their series "Memory allocation profiling" Kent Overstreet and
Suren Baghdasaryan have contributed a means of determining (via
/proc/allocinfo) whereabouts in the kernel memory is being
allocated: number of calls and amount of memory.
- Matthew Wilcox has provided the series "Various significant MM
patches" which does a number of rather unrelated things, but in
largely similar code sites.
- In his series "mm: page_alloc: freelist migratetype hygiene"
Johannes Weiner has fixed the page allocator's handling of
migratetype requests, with resulting improvements in compaction
efficiency.
- In the series "make the hugetlb migration strategy consistent"
Baolin Wang has fixed a hugetlb migration issue, which should
improve hugetlb allocation reliability.
- Liu Shixin has hit an I/O meltdown caused by readahead in a
memory-tight memcg. Addressed in the series "Fix I/O high when
memory almost met memcg limit".
- In the series "mm/filemap: optimize folio adding and splitting"
Kairui Song has optimized pagecache insertion, yielding ~10%
performance improvement in one test.
- Baoquan He has cleaned up and consolidated the early zone
initialization code in the series "mm/mm_init.c: refactor
free_area_init_core()".
- Baoquan has also redone some MM initializatio code in the series
"mm/init: minor clean up and improvement".
- MM helper cleanups from Christoph Hellwig in his series "remove
follow_pfn".
- More cleanups from Matthew Wilcox in the series "Various
page->flags cleanups".
- Vlastimil Babka has contributed maintainability improvements in the
series "memcg_kmem hooks refactoring".
- More folio conversions and cleanups in Matthew Wilcox's series:
"Convert huge_zero_page to huge_zero_folio"
"khugepaged folio conversions"
"Remove page_idle and page_young wrappers"
"Use folio APIs in procfs"
"Clean up __folio_put()"
"Some cleanups for memory-failure"
"Remove page_mapping()"
"More folio compat code removal"
- David Hildenbrand chipped in with "fs/proc/task_mmu: convert
hugetlb functions to work on folis".
- Code consolidation and cleanup work related to GUP's handling of
hugetlbs in Peter Xu's series "mm/gup: Unify hugetlb, part 2".
- Rick Edgecombe has developed some fixes to stack guard gaps in the
series "Cover a guard gap corner case".
- Jinjiang Tu has fixed KSM's behaviour after a fork+exec in the
series "mm/ksm: fix ksm exec support for prctl".
- Baolin Wang has implemented NUMA balancing for multi-size THPs.
This is a simple first-cut implementation for now. The series is
"support multi-size THP numa balancing".
- Cleanups to vma handling helper functions from Matthew Wilcox in
the series "Unify vma_address and vma_pgoff_address".
- Some selftests maintenance work from Dev Jain in the series
"selftests/mm: mremap_test: Optimizations and style fixes".
- Improvements to the swapping of multi-size THPs from Ryan Roberts
in the series "Swap-out mTHP without splitting".
- Kefeng Wang has significantly optimized the handling of arm64's
permission page faults in the series
"arch/mm/fault: accelerate pagefault when badaccess"
"mm: remove arch's private VM_FAULT_BADMAP/BADACCESS"
- GUP cleanups from David Hildenbrand in "mm/gup: consistently call
it GUP-fast".
- hugetlb fault code cleanups from Vishal Moola in "Hugetlb fault
path to use struct vm_fault".
- selftests build fixes from John Hubbard in the series "Fix
selftests/mm build without requiring "make headers"".
- Memory tiering fixes/improvements from Ho-Ren (Jack) Chuang in the
series "Improved Memory Tier Creation for CPUless NUMA Nodes".
Fixes the initialization code so that migration between different
memory types works as intended.
- David Hildenbrand has improved follow_pte() and fixed an errant
driver in the series "mm: follow_pte() improvements and acrn
follow_pte() fixes".
- David also did some cleanup work on large folio mapcounts in his
series "mm: mapcount for large folios + page_mapcount() cleanups".
- Folio conversions in KSM in Alex Shi's series "transfer page to
folio in KSM".
- Barry Song has added some sysfs stats for monitoring multi-size
THP's in the series "mm: add per-order mTHP alloc and swpout
counters".
- Some zswap cleanups from Yosry Ahmed in the series "zswap
same-filled and limit checking cleanups".
- Matthew Wilcox has been looking at buffer_head code and found the
documentation to be lacking. The series is "Improve buffer head
documentation".
- Multi-size THPs get more work, this time from Lance Yang. His
series "mm/madvise: enhance lazyfreeing with mTHP in madvise_free"
optimizes the freeing of these things.
- Kemeng Shi has added more userspace-visible writeback
instrumentation in the series "Improve visibility of writeback".
- Kemeng Shi then sent some maintenance work on top in the series
"Fix and cleanups to page-writeback".
- Matthew Wilcox reduces mmap_lock traffic in the anon vma code in
the series "Improve anon_vma scalability for anon VMAs". Intel's
test bot reported an improbable 3x improvement in one test.
- SeongJae Park adds some DAMON feature work in the series
"mm/damon: add a DAMOS filter type for page granularity access recheck"
"selftests/damon: add DAMOS quota goal test"
- Also some maintenance work in the series
"mm/damon/paddr: simplify page level access re-check for pageout"
"mm/damon: misc fixes and improvements"
- David Hildenbrand has disabled some known-to-fail selftests ni the
series "selftests: mm: cow: flag vmsplice() hugetlb tests as
XFAIL".
- memcg metadata storage optimizations from Shakeel Butt in "memcg:
reduce memory consumption by memcg stats".
- DAX fixes and maintenance work from Vishal Verma in the series
"dax/bus.c: Fixups for dax-bus locking""
* tag 'mm-stable-2024-05-17-19-19' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (426 commits)
memcg, oom: cleanup unused memcg_oom_gfp_mask and memcg_oom_order
selftests/mm: hugetlb_madv_vs_map: avoid test skipping by querying hugepage size at runtime
mm/hugetlb: add missing VM_FAULT_SET_HINDEX in hugetlb_wp
mm/hugetlb: add missing VM_FAULT_SET_HINDEX in hugetlb_fault
selftests: cgroup: add tests to verify the zswap writeback path
mm: memcg: make alloc_mem_cgroup_per_node_info() return bool
mm/damon/core: fix return value from damos_wmark_metric_value
mm: do not update memcg stats for NR_{FILE/SHMEM}_PMDMAPPED
selftests: cgroup: remove redundant enabling of memory controller
Docs/mm/damon/maintainer-profile: allow posting patches based on damon/next tree
Docs/mm/damon/maintainer-profile: change the maintainer's timezone from PST to PT
Docs/mm/damon/design: use a list for supported filters
Docs/admin-guide/mm/damon/usage: fix wrong schemes effective quota update command
Docs/admin-guide/mm/damon/usage: fix wrong example of DAMOS filter matching sysfs file
selftests/damon: classify tests for functionalities and regressions
selftests/damon/_damon_sysfs: use 'is' instead of '==' for 'None'
selftests/damon/_damon_sysfs: find sysfs mount point from /proc/mounts
selftests/damon/_damon_sysfs: check errors from nr_schemes file reads
mm/damon/core: initialize ->esz_bp from damos_quota_init_priv()
selftests/damon: add a test for DAMOS quota goal
...
124 lines
4.1 KiB
C
124 lines
4.1 KiB
C
/* SPDX-License-Identifier: GPL-2.0-only */
|
|
/*
|
|
* arch/arm/include/asm/pgtable-3level-hwdef.h
|
|
*
|
|
* Copyright (C) 2011 ARM Ltd.
|
|
* Author: Catalin Marinas <catalin.marinas@arm.com>
|
|
*/
|
|
#ifndef _ASM_PGTABLE_3LEVEL_HWDEF_H
|
|
#define _ASM_PGTABLE_3LEVEL_HWDEF_H
|
|
|
|
/*
|
|
* Hardware page table definitions.
|
|
*
|
|
* + Level 1/2 descriptor
|
|
* - common
|
|
*/
|
|
#define PUD_TABLE_BIT (_AT(pmdval_t, 1) << 1)
|
|
#define PMD_TYPE_MASK (_AT(pmdval_t, 3) << 0)
|
|
#define PMD_TYPE_FAULT (_AT(pmdval_t, 0) << 0)
|
|
#define PMD_TYPE_TABLE (_AT(pmdval_t, 3) << 0)
|
|
#define PMD_TYPE_SECT (_AT(pmdval_t, 1) << 0)
|
|
#define PMD_TABLE_BIT (_AT(pmdval_t, 1) << 1)
|
|
#define PMD_BIT4 (_AT(pmdval_t, 0))
|
|
#define PMD_DOMAIN(x) (_AT(pmdval_t, 0))
|
|
#define PMD_APTABLE_SHIFT (61)
|
|
#define PMD_APTABLE (_AT(pgdval_t, 3) << PGD_APTABLE_SHIFT)
|
|
#define PMD_PXNTABLE (_AT(pgdval_t, 1) << 59)
|
|
|
|
/*
|
|
* - section
|
|
*/
|
|
#define PMD_SECT_BUFFERABLE (_AT(pmdval_t, 1) << 2)
|
|
#define PMD_SECT_CACHEABLE (_AT(pmdval_t, 1) << 3)
|
|
#define PMD_SECT_USER (_AT(pmdval_t, 1) << 6) /* AP[1] */
|
|
#define PMD_SECT_AP2 (_AT(pmdval_t, 1) << 7) /* read only */
|
|
#define PMD_SECT_S (_AT(pmdval_t, 3) << 8)
|
|
#define PMD_SECT_AF (_AT(pmdval_t, 1) << 10)
|
|
#define PMD_SECT_nG (_AT(pmdval_t, 1) << 11)
|
|
#define PMD_SECT_PXN (_AT(pmdval_t, 1) << 53)
|
|
#define PMD_SECT_XN (_AT(pmdval_t, 1) << 54)
|
|
#define PMD_SECT_AP_WRITE (_AT(pmdval_t, 0))
|
|
#define PMD_SECT_AP_READ (_AT(pmdval_t, 0))
|
|
#define PMD_SECT_AP1 (_AT(pmdval_t, 1) << 6)
|
|
#define PMD_SECT_TEX(x) (_AT(pmdval_t, 0))
|
|
|
|
/*
|
|
* AttrIndx[2:0] encoding (mapping attributes defined in the MAIR* registers).
|
|
*/
|
|
#define PMD_SECT_UNCACHED (_AT(pmdval_t, 0) << 2) /* strongly ordered */
|
|
#define PMD_SECT_BUFFERED (_AT(pmdval_t, 1) << 2) /* normal non-cacheable */
|
|
#define PMD_SECT_WT (_AT(pmdval_t, 2) << 2) /* normal inner write-through */
|
|
#define PMD_SECT_WB (_AT(pmdval_t, 3) << 2) /* normal inner write-back */
|
|
#define PMD_SECT_WBWA (_AT(pmdval_t, 7) << 2) /* normal inner write-alloc */
|
|
#define PMD_SECT_CACHE_MASK (_AT(pmdval_t, 7) << 2)
|
|
|
|
/*
|
|
* + Level 3 descriptor (PTE)
|
|
*/
|
|
#define PTE_TYPE_MASK (_AT(pteval_t, 3) << 0)
|
|
#define PTE_TYPE_FAULT (_AT(pteval_t, 0) << 0)
|
|
#define PTE_TYPE_PAGE (_AT(pteval_t, 3) << 0)
|
|
#define PTE_TABLE_BIT (_AT(pteval_t, 1) << 1)
|
|
#define PTE_BUFFERABLE (_AT(pteval_t, 1) << 2) /* AttrIndx[0] */
|
|
#define PTE_CACHEABLE (_AT(pteval_t, 1) << 3) /* AttrIndx[1] */
|
|
#define PTE_AP2 (_AT(pteval_t, 1) << 7) /* AP[2] */
|
|
#define PTE_EXT_SHARED (_AT(pteval_t, 3) << 8) /* SH[1:0], inner shareable */
|
|
#define PTE_EXT_AF (_AT(pteval_t, 1) << 10) /* Access Flag */
|
|
#define PTE_EXT_NG (_AT(pteval_t, 1) << 11) /* nG */
|
|
#define PTE_EXT_PXN (_AT(pteval_t, 1) << 53) /* PXN */
|
|
#define PTE_EXT_XN (_AT(pteval_t, 1) << 54) /* XN */
|
|
|
|
/*
|
|
* 40-bit physical address supported.
|
|
*/
|
|
#define PHYS_MASK_SHIFT (40)
|
|
#define PHYS_MASK ((1ULL << PHYS_MASK_SHIFT) - 1)
|
|
|
|
#ifndef CONFIG_CPU_TTBR0_PAN
|
|
/*
|
|
* TTBR0/TTBR1 split (PAGE_OFFSET):
|
|
* 0x40000000: T0SZ = 2, T1SZ = 0 (not used)
|
|
* 0x80000000: T0SZ = 0, T1SZ = 1
|
|
* 0xc0000000: T0SZ = 0, T1SZ = 2
|
|
*
|
|
* Only use this feature if PHYS_OFFSET <= PAGE_OFFSET, otherwise
|
|
* booting secondary CPUs would end up using TTBR1 for the identity
|
|
* mapping set up in TTBR0.
|
|
*/
|
|
#if defined CONFIG_VMSPLIT_2G
|
|
#define TTBR1_OFFSET 16 /* skip two L1 entries */
|
|
#elif defined CONFIG_VMSPLIT_3G
|
|
#define TTBR1_OFFSET (4096 * (1 + 3)) /* only L2, skip pgd + 3*pmd */
|
|
#else
|
|
#define TTBR1_OFFSET 0
|
|
#endif
|
|
|
|
#define TTBR1_SIZE (((PAGE_OFFSET >> 30) - 1) << 16)
|
|
#else
|
|
/*
|
|
* With CONFIG_CPU_TTBR0_PAN enabled, TTBR1 is only used during uaccess
|
|
* disabled regions when TTBR0 is disabled.
|
|
*/
|
|
#define TTBR1_OFFSET 0 /* pointing to swapper_pg_dir */
|
|
#define TTBR1_SIZE 0 /* TTBR1 size controlled via TTBCR.T0SZ */
|
|
#endif
|
|
|
|
/*
|
|
* TTBCR register bits.
|
|
*/
|
|
#define TTBCR_EAE (1 << 31)
|
|
#define TTBCR_IMP (1 << 30)
|
|
#define TTBCR_SH1_MASK (3 << 28)
|
|
#define TTBCR_ORGN1_MASK (3 << 26)
|
|
#define TTBCR_IRGN1_MASK (3 << 24)
|
|
#define TTBCR_EPD1 (1 << 23)
|
|
#define TTBCR_A1 (1 << 22)
|
|
#define TTBCR_T1SZ_MASK (7 << 16)
|
|
#define TTBCR_SH0_MASK (3 << 12)
|
|
#define TTBCR_ORGN0_MASK (3 << 10)
|
|
#define TTBCR_IRGN0_MASK (3 << 8)
|
|
#define TTBCR_EPD0 (1 << 7)
|
|
#define TTBCR_T0SZ_MASK (7 << 0)
|
|
|
|
#endif
|