Loading...
--- /dev/null
+++ libmalloc/libmalloc-283.100.5/tests/MallocBenchTest/BMALLOC/bmalloc/ChangeLog
@@ -0,0 +1,9969 @@
+2018-11-21 Dominik Infuehr <dinfuehr@igalia.com>
+
+ Enable JIT on ARM/Linux
+ https://bugs.webkit.org/show_bug.cgi?id=191548
+
+ Reviewed by Yusuke Suzuki.
+
+ * bmalloc/IsoPageInlines.h:
+ (bmalloc::IsoPage<Config>::startAllocating):
+
+2018-11-01 Jiewen Tan <jiewen_tan@apple.com>
+
+ Replace CommonRandom SPI with API
+ https://bugs.webkit.org/show_bug.cgi?id=191178
+ <rdar://problem/45722391>
+
+ Reviewed by Brent Fulgham.
+
+ * bmalloc/CryptoRandom.cpp:
+ (bmalloc::ARC4RandomNumberGenerator::stir):
+
+2018-10-29 Mark Lam <mark.lam@apple.com>
+
+ Correctly detect string overflow when using the 'Function' constructor.
+ https://bugs.webkit.org/show_bug.cgi?id=184883
+ <rdar://problem/36320331>
+
+ Reviewed by Saam Barati.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate):
+ (bmalloc::Allocator::tryReallocate):
+ (bmalloc::Allocator::reallocateImpl):
+ * bmalloc/Allocator.h:
+ * bmalloc/Cache.h:
+ (bmalloc::Cache::tryReallocate):
+ * bmalloc/DebugHeap.cpp:
+ (bmalloc::DebugHeap::realloc):
+ * bmalloc/DebugHeap.h:
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::tryRealloc):
+
+2018-10-25 Ross Kirsling <ross.kirsling@sony.com>
+
+ Cleanup: inline constexpr is redundant as constexpr implies inline
+ https://bugs.webkit.org/show_bug.cgi?id=190819
+
+ Reviewed by Mark Lam.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::max):
+ (bmalloc::min):
+ (bmalloc::mask):
+ (bmalloc::test):
+ (bmalloc::isPowerOfTwo):
+ (bmalloc::roundDownToMultipleOf):
+ (bmalloc::sizeOf):
+ (bmalloc::bitCount):
+ (bmalloc::log2):
+ * bmalloc/Bits.h:
+ (bmalloc::bitsArrayLength):
+ * bmalloc/Sizes.h:
+ (bmalloc::Sizes::maskSizeClass):
+
+2018-10-24 Alexey Proskuryakov <ap@apple.com>
+
+ Add BPLATFORM(IOS_FAMILY)
+ https://bugs.webkit.org/show_bug.cgi?id=190878
+
+ Reviewed by Saam Barati.
+
+ * bmalloc/AvailableMemory.cpp:
+ (bmalloc::memorySizeAccordingToKernel):
+ (bmalloc::computeAvailableMemory):
+ * bmalloc/AvailableMemory.h:
+ (bmalloc::isUnderMemoryPressure):
+ * bmalloc/BPlatform.h:
+ * bmalloc/Gigacage.h:
+ * bmalloc/Logging.cpp:
+ (bmalloc::logVMFailure):
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmPageSizePhysical):
+ * bmalloc/bmalloc.h:
+ * bmalloc/darwin/MemoryStatusSPI.h:
+
+2018-10-12 Ryan Haddad <ryanhaddad@apple.com>
+
+ Unreviewed, rolling out r237063.
+
+ Caused layout test fast/dom/Window/window-postmessage-clone-
+ deep-array.html to fail on macOS and iOS Debug bots.
+
+ Reverted changeset:
+
+ "[JSC] Remove gcc warnings on mips and armv7"
+ https://bugs.webkit.org/show_bug.cgi?id=188598
+ https://trac.webkit.org/changeset/237063
+
+2018-10-11 Guillaume Emont <guijemont@igalia.com>
+
+ [JSC] Remove gcc warnings on mips and armv7
+ https://bugs.webkit.org/show_bug.cgi?id=188598
+
+ Reviewed by Mark Lam.
+
+ Add bitwise_cast (from WTF) and use it instead of reinterpret_cast in
+ a couple places where reinterpret_cast triggers a warning about
+ alignment even though we know that alignment is correct.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::bitwise_cast): Copied from WTF/wtf/StdLibextras.h
+ * bmalloc/IsoDirectoryPageInlines.h:
+ (bmalloc::IsoDirectoryPage<Config>::pageFor):
+ * bmalloc/IsoPageInlines.h:
+ (bmalloc::IsoPage<Config>::startAllocating):
+
+2018-10-03 Dan Bernstein <mitz@apple.com>
+
+ bmalloc part of [Xcode] Update some build settings as recommended by Xcode 10
+ https://bugs.webkit.org/show_bug.cgi?id=190250
+
+ Reviewed by Alex Christensen.
+
+ * Configurations/Base.xcconfig: Enabled CLANG_WARN_COMMA, CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS,
+ and CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF.
+
+ * bmalloc.xcodeproj/project.pbxproj: Let Xcode update LastUpgradeCheck.
+
+2018-09-25 Alex Christensen <achristensen@webkit.org>
+
+ Allow for suffixes to com.apple.WebKit.WebContent
+ https://bugs.webkit.org/show_bug.cgi?id=189972
+
+ Reviewed by Chris Dumez.
+
+ * bmalloc/ProcessCheck.mm:
+ (bmalloc::gigacageEnabledForProcess):
+
+2018-09-24 Fujii Hironori <Hironori.Fujii@sony.com>
+
+ Rename WTF_COMPILER_GCC_OR_CLANG to WTF_COMPILER_GCC_COMPATIBLE
+ https://bugs.webkit.org/show_bug.cgi?id=189733
+
+ Reviewed by Michael Catanzaro.
+
+ * bmalloc/BCompiler.h:
+
+2018-08-27 Keith Rollin <krollin@apple.com>
+
+ Unreviewed build fix -- disable LTO for production builds
+
+ * Configurations/Base.xcconfig:
+
+2018-08-27 Keith Rollin <krollin@apple.com>
+
+ Build system support for LTO
+ https://bugs.webkit.org/show_bug.cgi?id=187785
+ <rdar://problem/42353132>
+
+ Reviewed by Dan Bernstein.
+
+ Update Base.xcconfig and DebugRelease.xcconfig to optionally enable
+ LTO.
+
+ * Configurations/Base.xcconfig:
+ * Configurations/DebugRelease.xcconfig:
+
+2018-08-16 Tomas Popela <tpopela@redhat.com>
+
+ bmalloc: Coverity scan issues
+ https://bugs.webkit.org/show_bug.cgi?id=186763
+
+ Reviewed by Saam Barati.
+
+ * bmalloc/DebugHeap.h: Initialize the m_pageSize variable.
+ * bmalloc/IsoTLS.cpp:
+ (bmalloc::IsoTLS::ensureEntries): Check the return value of
+ pthread_key_create return().
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmPageSize): Correctly check the return value of sysconf().
+
+2018-07-27 Mark Lam <mark.lam@apple.com>
+
+ Initialize bmalloc::DebugHeap::m_pageSize for non-Darwin builds.
+ https://bugs.webkit.org/show_bug.cgi?id=188132
+ <rdar://problem/40401599>
+
+ Reviewed by Saam Barati.
+
+ * bmalloc/DebugHeap.cpp:
+ (bmalloc::DebugHeap::DebugHeap):
+
+2018-07-27 Saam Barati <sbarati@apple.com>
+
+ Explicitly handle memlimit_active < 0
+ https://bugs.webkit.org/show_bug.cgi?id=188125
+
+ Reviewed by Mark Lam.
+
+ This may come up during development when someone wants the limit
+ to be "infinite".
+
+ * bmalloc/AvailableMemory.cpp:
+ (bmalloc::jetsamLimit):
+
+2018-07-27 Saam Barati <sbarati@apple.com>
+
+ Use SPI to compute the jetsam limit on iOS instead of hardcoding 840MB
+ https://bugs.webkit.org/show_bug.cgi?id=188091
+ <rdar://problem/42647697>
+
+ Reviewed by Simon Fraser.
+
+ We want bmalloc to dynamically adapt to the jetsam limit of the process
+ it's running in. WTF::ramSize() is based off bmalloc's availableMemory,
+ so it will now reflect the result of the real jetsam limit when we can
+ read it.
+
+ Reading the jetsam limit requires an entitlement, so this patch opts in
+ the WebContent/Storage/Network processes. We fall back to 840MB (the
+ old hard coded value) when the SPI call fails (e.g, when we're in a
+ process without the proper entitlement).
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/AvailableMemory.cpp:
+ (bmalloc::jetsamLimit):
+ (bmalloc::computeAvailableMemory):
+ * bmalloc/darwin/MemoryStatusSPI.h: Added.
+
+2018-07-24 Saam Barati <sbarati@apple.com>
+
+ Revert back to using phys_footprint to calculate isUnderMemoryPressure()
+ https://bugs.webkit.org/show_bug.cgi?id=187919
+ <rdar://problem/42552888>
+
+ Reviewed by Simon Fraser.
+
+ Currently on iOS, bmalloc will run the scavenger more frequently when it detects
+ that the process is under memory pressure. However, it only uses bmalloc's
+ own footprint as a percentage of the HW available memory to determine if
+ the process is under memory pressure. This is a change I recently made
+ in an effort to run the scavenger less when bmalloc wasn't contributing
+ to the dirty footprint in the process. However, this fails to run the
+ scavenger eagerly when the process in question has a heap split
+ between a lot of dirty bmalloc memory as well as a lot of dirty memory
+ from elsewhere. We also have evidence that we may have increased jetsams
+ in the Web Content process. Since my original change was not a measurable
+ speedup, this patch reverts isUnderMemoryPressure() to its previous
+ behavior of using phys_footprint to determine if 75% of the available
+ HW memory is being used.
+
+ * bmalloc/AvailableMemory.cpp:
+ (bmalloc::memoryStatus):
+
+2019-07-12 Michael Saboff <msaboff@apple.com>
+
+ Disable IsoHeaps when Gigacage is off
+ https://bugs.webkit.org/show_bug.cgi?id=187160
+
+ Reviewed by Saam Barati.
+
+ Relanding change sets 233547 and 233550 with the added fix that Gigacage is also
+ enabled for DumpRenderTree.
+
+ Updated determineMallocFallbackState to base enabling of Iso Heaps on Gigacage
+ being enabled. We do this because if Gigacage is disabled, it may be due to lack
+ of address space.
+
+ To work around a compiler issue uncovered by the change above, I added explicit
+ instantiation of PerThread's static variables. Defined the same explicit
+ instantiated static variables with export scope in the new file PerThread.cpp
+ to eliminate separate variables allocations in each linked framework / library.
+
+ * CMakeLists.txt:
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/IsoTLS.cpp:
+ (bmalloc::IsoTLS::determineMallocFallbackState):
+ * bmalloc/PerThread.cpp: Added.
+ * bmalloc/PerThread.h:
+ * bmalloc/ProcessCheck.mm:
+ (bmalloc::gigacageEnabledForProcess):
+
+2018-07-09 Commit Queue <commit-queue@webkit.org>
+
+ Unreviewed, rolling out r233547 and r233550.
+ https://bugs.webkit.org/show_bug.cgi?id=187497
+
+ Introduced flakiness for media/fullscreen-* tests on mac-wk1
+ (Requested by ryanhaddad on #webkit).
+
+ Reverted changesets:
+
+ "Disable IsoHeaps when Gigacage is off"
+ https://bugs.webkit.org/show_bug.cgi?id=187160
+ https://trac.webkit.org/changeset/233547
+
+ "Build fix (r233547): Disable IsoHeaps when Gigacage is off"
+ https://bugs.webkit.org/show_bug.cgi?id=187160
+ https://trac.webkit.org/changeset/233550
+
+2018-07-05 David Kilzer <ddkilzer@apple.com>
+
+ Build fix (r233547): Disable IsoHeaps when Gigacage is off
+ <https://webkit.org/b/187160>
+
+ * bmalloc/PerThread.cpp: Add #if !HAVE_PTHREAD_MACHDEP_H/#endif
+ around variables only used when that macro is 0. Include what
+ you use: Cache.h and Heap.h.
+ * bmalloc/PerThread.h: Include <memory> for std::once_flag.
+
+2018-07-05 Michael Saboff <msaboff@apple.com>
+
+ Disable IsoHeaps when Gigacage is off
+ https://bugs.webkit.org/show_bug.cgi?id=187160
+
+ Reviewed by Saam Barati.
+
+ Updated determineMallocFallbackState to base enabling of Iso Heaps on Gigacage
+ being enabled. We do this because if Gigacage is disabled, it may be due to lack
+ of address space.
+
+ To work around a compiler issue uncovered by the change above, I added explicit
+ instantiation of PerThread's static variables. Defined the same explicit
+ instantiated static variables with export scope in the new file PerThread.cpp
+ to eliminate separate variables allocations in each linked framework / library.
+
+ * CMakeLists.txt:
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/IsoTLS.cpp:
+ (bmalloc::IsoTLS::determineMallocFallbackState):
+ * bmalloc/PerThread.cpp: Added.
+ * bmalloc/PerThread.h:
+
+2018-07-04 Tim Horton <timothy_horton@apple.com>
+
+ Introduce PLATFORM(IOSMAC)
+ https://bugs.webkit.org/show_bug.cgi?id=187315
+
+ Reviewed by Dan Bernstein.
+
+ * Configurations/Base.xcconfig:
+
+2018-06-29 Ryan Haddad <ryanhaddad@apple.com>
+
+ Unreviewed, rolling out r233347.
+
+ Causes crashes during WK1 tests.
+
+ Reverted changeset:
+
+ "Disable IsoHeaps when Gigacage is off"
+ https://bugs.webkit.org/show_bug.cgi?id=187160
+ https://trac.webkit.org/changeset/233347
+
+2018-06-28 Michael Saboff <msaboff@apple.com>
+
+ Disable IsoHeaps when Gigacage is off
+ https://bugs.webkit.org/show_bug.cgi?id=187160
+
+ Reviewed by Saam Barati.
+
+ If Gigacage is disabled, it may be due to lack of address space.
+ Therefore we should also turn off IsoHeaps since it uses more virtual
+ address space as well.
+
+ * bmalloc/IsoTLS.cpp:
+ (bmalloc::IsoTLS::determineMallocFallbackState):
+
+2018-06-27 Simon Fraser <simon.fraser@apple.com>
+
+ https://hackernoon.com/ uses lots of layer backing store
+ https://bugs.webkit.org/show_bug.cgi?id=186909
+ rdar://problem/40257540
+
+ Reviewed by Tim Horton.
+
+ Drive-by typo fix.
+
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::dumpStats):
+
+2018-06-26 Saam Barati <sbarati@apple.com>
+
+ Unreviewed followup. Fix the watchos build after r233192.
+
+ This patch also correct the changelog entry below to have the correct
+ bug and title info.
+
+ * bmalloc/ProcessCheck.mm:
+
+2018-06-26 Saam Barati <sbarati@apple.com>
+
+ Switch to system malloc on iOS when nano malloc is disabled
+ https://bugs.webkit.org/show_bug.cgi?id=186322
+ <rdar://problem/41140257>
+
+ Reviewed by Keith Miller.
+
+ We have evidence showing that processes with small heaps using the
+ JS API are more space efficient when using system malloc. Our main
+ hypothesis as to why this is, is that when dealing with small heaps,
+ one malloc can be more efficient at optimizing memory usage than
+ two mallocs.
+
+ * bmalloc/BPlatform.h:
+ * bmalloc/Environment.cpp:
+ (bmalloc::isNanoMallocEnabled):
+ (bmalloc::Environment::computeIsDebugHeapEnabled):
+ * bmalloc/ProcessCheck.h:
+ (bmalloc::shouldProcessUnconditionallyUseBmalloc):
+ * bmalloc/ProcessCheck.mm:
+ (bmalloc::shouldProcessUnconditionallyUseBmalloc):
+
+2018-06-24 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ [bmalloc][Linux] Remove static initializers for PerProcess<>::s_object
+ https://bugs.webkit.org/show_bug.cgi?id=186966
+
+ Reviewed by Anders Carlsson.
+
+ chrome/tools/linux/dump-static-initializers.py can dump static initializers
+ in the binary and we found that PerProcess<>::s_object initialization is done
+ by static initializers in GCC + Linux environments. The example is the following.
+
+ Scavenger.cpp (initializer offset 0x38c210 size 0x3e)
+ _GLOBAL__sub_I_Scavenger.cpp+0x1e
+ _GLOBAL__sub_I_Scavenger.cpp+0x2d
+ _GLOBAL__sub_I_Scavenger.cpp+0x3c
+ _GLOBAL__sub_I_Scavenger.cpp+0xf
+ guard variable for bmalloc::PerProcess<bmalloc::AllIsoHeaps>::s_object@@Base-0x3f0d8
+ guard variable for bmalloc::PerProcess<bmalloc::Environment>::s_object@@Base-0x3f0e8
+ guard variable for bmalloc::PerProcess<bmalloc::PerHeapKind<bmalloc::Heap> >::s_object@@Base-0x3c600
+ guard variable for bmalloc::PerProcess<bmalloc::Scavenger>::s_object@@Base-0x38ce8
+
+ We can remove this by initializing `nullptr`, which leads to constexpr initialization.
+ After this change, Linux JSCOnly libJavaScriptCore.so has no static initializers.
+
+ * bmalloc/PerProcess.h:
+
+2018-06-09 Dan Bernstein <mitz@apple.com>
+
+ [Xcode] Clean up and modernize some build setting definitions
+ https://bugs.webkit.org/show_bug.cgi?id=186463
+
+ Reviewed by Sam Weinig.
+
+ * Configurations/Base.xcconfig: Removed definition for macOS 10.11.
+ * Configurations/DebugRelease.xcconfig: Ditto.
+
+2018-06-07 Darin Adler <darin@apple.com>
+
+ [Cocoa] Turn on ARC for the single Objective-C++ source file in bmalloc
+ https://bugs.webkit.org/show_bug.cgi?id=186398
+
+ Reviewed by Saam Barati.
+
+ * Configurations/Base.xcconfig: Turn on ARC.
+ * bmalloc/ProcessCheck.mm:
+ (bmalloc::gigacageEnabledForProcess): Removed the globals from this function,
+ since it's only called once. If it was called more than once, we could optimize
+ that with a single boolean global rather than two strings and two booleans.
+
+2018-06-07 David Kilzer <ddkilzer@apple.com>
+
+ bmalloc: Fix 'noreturn' warnings when compiling with -std=gnu++17
+ <https://webkit.org/b/186400>
+
+ Reviewed by Saam Barati.
+
+ Fixes the following warnings when compiling with gnu++17:
+
+ Source/bmalloc/bmalloc/Scavenger.cpp:363:1: error: function 'threadRunLoop' could be declared with attribute 'noreturn' [-Werror,-Wmissing-noreturn]
+ {
+ ^
+ Source/bmalloc/bmalloc/Scavenger.cpp:358:1: error: function 'threadEntryPoint' could be declared with attribute 'noreturn' [-Werror,-Wmissing-noreturn]
+ {
+ ^
+
+ * bmalloc/BCompiler.h:
+ (BCOMPILER): Add support for the BCOMPILER() macro, then add
+ BCOMPILER(GCC_OR_CLANG). Taken from Source/WTF/wtf/Compiler.h.
+ (BNO_RETURN): Implement 'norerturn' support using the new
+ BCOMPILER() macros. Taken from Source/WTF/wtf/Compiler.h.
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::Scavenger::threadRunLoop): Remove the workaround that
+ tricked older compilers into thinking the while() loop wasn't
+ infinite.
+ * bmalloc/Scavenger.h:
+ (bmalloc::Scavenger::threadEntryPoint): Add BNO_RETURN attribute.
+ (bmalloc::Scavenger::threadRunLoop): Ditto.
+
+2018-05-29 Saam Barati <sbarati@apple.com>
+
+ JSC should put bmalloc's scavenger into mini mode
+ https://bugs.webkit.org/show_bug.cgi?id=185988
+
+ Reviewed by Michael Saboff.
+
+ We expose an API for putting bmalloc into mini mode. All that means now
+ is that we'll run the scavenger more aggressively.
+
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::Scavenger::enableMiniMode):
+ (bmalloc::Scavenger::threadRunLoop):
+ * bmalloc/Scavenger.h:
+ * bmalloc/Sizes.h:
+ * bmalloc/bmalloc.cpp:
+ (bmalloc::api::enableMiniMode):
+ * bmalloc/bmalloc.h:
+
+2018-05-29 Geoffrey Garen <ggaren@apple.com>
+
+ Fixed the bmalloc build
+ https://bugs.webkit.org/show_bug.cgi?id=186025
+
+ Reviewed by Sam Weinig.
+
+ * bmalloc.xcodeproj/project.pbxproj: Link Foundation because the
+ gigacage check needs it.
+
+2018-05-23 Antti Koivisto <antti@apple.com>
+
+ Increase the simulated memory size on PLATFORM(IOS_SIMULATOR) from 512MB to 1024MB
+ https://bugs.webkit.org/show_bug.cgi?id=185908
+
+ Reviewed by Geoffrey Garen.
+
+ We don't support 512MB devices anymore. This will make the simulator behave more
+ like a real device.
+
+ * bmalloc/AvailableMemory.cpp:
+ (bmalloc::memorySizeAccordingToKernel):
+
+ Factor to a function.
+ Don't use availableMemoryGuess for the simulator value as it is not a guess.
+
+ (bmalloc::computeAvailableMemory):
+
+ Apply the same adjustments to the simulated value too.
+
+2018-05-22 Ryan Haddad <ryanhaddad@apple.com>
+
+ Unreviewed, rolling out r232052.
+
+ Breaks internal builds.
+
+ Reverted changeset:
+
+ "Use more C++17"
+ https://bugs.webkit.org/show_bug.cgi?id=185176
+ https://trac.webkit.org/changeset/232052
+
+2018-05-22 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ Define GIGACAGE_ALLOCATION_CAN_FAIL on Linux
+ https://bugs.webkit.org/show_bug.cgi?id=183329
+
+ Reviewed by Michael Catanzaro.
+
+ We specify `GIGACAGE_ALLOCATION_CAN_FAIL 1` in Linux since
+ Linux can fail to `mmap` if `vm.overcommit_memory = 2`.
+ Users can enable Gigacage if users enable overcommit_memory.
+
+ * bmalloc/Gigacage.h:
+
+2018-05-21 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ Use more C++17
+ https://bugs.webkit.org/show_bug.cgi?id=185176
+
+ Reviewed by JF Bastien.
+
+ Add BNO_RETURN.
+
+ * Configurations/Base.xcconfig:
+ * bmalloc/BCompiler.h:
+ * bmalloc/Scavenger.h:
+
+2018-05-06 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ [JSC] Remove "using namespace std;" from JSC, bmalloc, WTF
+ https://bugs.webkit.org/show_bug.cgi?id=185362
+
+ Reviewed by Sam Weinig.
+
+ * bmalloc/Allocator.cpp:
+ * bmalloc/Deallocator.cpp:
+
+2018-05-03 Filip Pizlo <fpizlo@apple.com>
+
+ Strings should not be allocated in a gigacage
+ https://bugs.webkit.org/show_bug.cgi?id=185218
+
+ Reviewed by Saam Barati.
+
+ This removes the string gigacage.
+
+ Putting strings in a gigacage prevents read gadgets. The other things that get to be in gigacages
+ are there to prevent read-write gadgets.
+
+ Also, putting strings in a gigacage seems to have been a bigger regression than putting other
+ things in gigacages.
+
+ Therefore, to maximize the benefit/cost ratio of gigacages, we should evict strings from them. If
+ we want to throw away perf for security, there are more beneficial things to sacrifice.
+
+ * bmalloc/Gigacage.h:
+ (Gigacage::name):
+ (Gigacage::basePtr):
+ (Gigacage::size):
+ (Gigacage::forEachKind):
+ * bmalloc/HeapKind.h:
+ (bmalloc::isGigacage):
+ (bmalloc::gigacageKind):
+ (bmalloc::heapKind):
+ (bmalloc::isActiveHeapKindAfterEnsuringGigacage):
+ (bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):
+
+2018-04-30 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ Use WordLock instead of std::mutex for Threading
+ https://bugs.webkit.org/show_bug.cgi?id=185121
+
+ Reviewed by Geoffrey Garen.
+
+ Add constexpr to explicitly describe that bmalloc::Mutex constructor is constexpr.
+
+ * bmalloc/Mutex.h:
+
+2018-04-23 Ting-Wei Lan <lantw44@gmail.com>
+
+ Include stdio.h before using stderr
+ https://bugs.webkit.org/show_bug.cgi?id=184872
+
+ Reviewed by Yusuke Suzuki.
+
+ * bmalloc/PerProcess.cpp:
+ * bmalloc/Scavenger.cpp:
+
+2018-04-21 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ Unreviewed, follow-up patch after r230474
+ https://bugs.webkit.org/show_bug.cgi?id=166684
+
+ Add "JavaScriptCore" to Darwin name. And use short name "BMScavenger"
+ for Linux since adding "JavaScriptCore" makes the name too long for Linux.
+
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::Scavenger::threadRunLoop):
+
+2018-04-18 Jer Noble <jer.noble@apple.com>
+
+ Don't put build products into WK_ALTERNATE_WEBKIT_SDK_PATH for engineering builds
+ https://bugs.webkit.org/show_bug.cgi?id=184762
+
+ Reviewed by Dan Bernstein.
+
+ * Configurations/Base.xcconfig:
+
+2018-04-20 Daniel Bates <dabates@apple.com>
+
+ Remove code for compilers that did not support NSDMI for aggregates
+ https://bugs.webkit.org/show_bug.cgi?id=184599
+
+ Reviewed by Per Arne Vollan.
+
+ Remove workaround for earlier Visual Studio versions that did not support non-static data
+ member initializers (NSDMI) for aggregates. We have since updated all the build.webkit.org
+ and EWS bots to a newer version that supports this feature.
+
+ * bmalloc/BPlatform.h:
+ * bmalloc/List.h:
+ (bmalloc::ListNode::ListNode): Deleted.
+ (bmalloc::List::iterator::iterator): Deleted.
+
+2018-04-19 David Kilzer <ddkilzer@apple.com>
+
+ Enable Objective-C weak references
+ <https://webkit.org/b/184789>
+ <rdar://problem/39571716>
+
+ Reviewed by Dan Bernstein.
+
+ * Configurations/Base.xcconfig:
+ (CLANG_ENABLE_OBJC_WEAK): Enable.
+
+2018-04-12 Saam Barati <sbarati@apple.com>
+
+ Lessen partial scavenge interval on x86-64
+ https://bugs.webkit.org/show_bug.cgi?id=184577
+
+ Rubber-stamped by Filip Pizlo.
+
+ I initially made the scavenge interval longer because I had thought the
+ shorter interval caused a JetStream regression. I was mistaken though.
+ I was looking at the wrong commit range when analyzing perf data.
+
+ This patch shortens the interval, but still keeps x86-64 50% longer than
+ other architectures. We know that scavenging frequently on Mac is less
+ important to overall system performance than it is on iOS.
+
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::Scavenger::threadRunLoop):
+
+2018-04-12 Saam Barati <sbarati@apple.com>
+
+ Raise the partial scavenge interval even more on x86-64
+ https://bugs.webkit.org/show_bug.cgi?id=184551
+
+ Rubber-stamped by Filip Pizlo.
+
+ The JetStream regression didn't recover from my previous patch.
+ This is another attempt to get it to recover perf.
+
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::Scavenger::threadRunLoop):
+
+2018-04-11 Saam Barati <sbarati@apple.com>
+
+ raise partial scavenge interval on x86-64
+ https://bugs.webkit.org/show_bug.cgi?id=184521
+
+ Rubber-stamped by Filip Pizlo.
+
+ This patch is an attempt to recover the 1-3% JetStream regression
+ my initial partial scavenging patch introduced on some Macs.
+
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::Scavenger::threadRunLoop):
+
+2018-04-10 Saam Barati <sbarati@apple.com>
+
+ IsoHeapImpl::scavenge* needs to grab the lock
+ https://bugs.webkit.org/show_bug.cgi?id=184461
+
+ Reviewed by Filip Pizlo.
+
+ Another thread could be modifying the linked list that the scavenge* methods traverse.
+
+ * bmalloc/IsoHeapImplInlines.h:
+ (bmalloc::IsoHeapImpl<Config>::scavenge):
+ (bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark):
+
+2018-04-10 Saam Barati <sbarati@apple.com>
+
+ bmalloc should do partial scavenges more frequently
+ https://bugs.webkit.org/show_bug.cgi?id=184176
+
+ Reviewed by Filip Pizlo.
+
+ This patch adds the ability for bmalloc to do a partial scavenge.
+ bmalloc will now do a partial scavenge with some frequency even
+ when the heap is growing.
+
+ For Heap, this means tracking the high water mark of where the Heap
+ has allocated since the last scavenge. Partial scavenging is just
+ decommitting entries in the LargeMap that are past this high water
+ mark. Because we allocate in first fit order out of LargeMap, tracking
+ the high water mark is a good heuristic of how much memory a partial
+ scavenge should decommit.
+
+ For IsoHeaps, each IsoDirectory also keeps track of its high water mark
+ for the furthest page it allocates into. Similar to Heap, we scavenge pages
+ past that high water mark. IsoHeapImpl then tracks the high water mark
+ for the IsoDirectory it allocates into. We then scavenge all directories
+ including and past the directory high water mark. This includes scavenging
+ the inline directory when its the only thing we allocate out of since
+ the last scavenge.
+
+ This patch also adds some other capabilities to bmalloc:
+
+ Heaps and IsoHeaps now track how much memory is freeable. Querying
+ this number is now cheap.
+
+ Heaps no longer hold the global lock when decommitting large ranges.
+ Instead, that range is just marked as non eligible to be allocated.
+ Then, without the lock held, the scavenger will decommit those ranges.
+ Once this is done, the scavenger will then reacquire the lock and mark
+ these ranges as eligible. This lessens lock contention between the
+ scavenger and the allocation slow path since threads that are taking an
+ allocation slow path can now allocate concurrently to the scavenger's
+ decommits. The main consideration in adding this functionality is that
+ a large allocation may fail while the scavenger is in the process of
+ decommitting memory. When the Heap fails to allocate a large range when
+ the scavenger is in the middle of a decommit, Heap will wait for the
+ Scavenger to finish and then it will try to allocate a large range again.
+
+ Decommitting from Heap now aggregates the ranges to decommit and tries to
+ merge them together to lower the number of calls to vmDeallocatePhysicalPages.
+ This is analogous to what IsoHeaps already do.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::tryAllocate):
+ (bmalloc::Allocator::allocateImpl):
+ (bmalloc::Allocator::reallocate):
+ (bmalloc::Allocator::refillAllocatorSlowCase):
+ (bmalloc::Allocator::allocateLarge):
+ * bmalloc/BulkDecommit.h: Added.
+ (bmalloc::BulkDecommit::addEager):
+ (bmalloc::BulkDecommit::addLazy):
+ (bmalloc::BulkDecommit::processEager):
+ (bmalloc::BulkDecommit::processLazy):
+ (bmalloc::BulkDecommit::add):
+ (bmalloc::BulkDecommit::process):
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::scavenge):
+ (bmalloc::Deallocator::processObjectLog):
+ (bmalloc::Deallocator::deallocateSlowCase):
+ * bmalloc/Deallocator.h:
+ (bmalloc::Deallocator::lineCache):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::freeableMemory):
+ (bmalloc::Heap::markAllLargeAsEligibile):
+ (bmalloc::Heap::decommitLargeRange):
+ (bmalloc::Heap::scavenge):
+ (bmalloc::Heap::scavengeToHighWatermark):
+ (bmalloc::Heap::deallocateLineCache):
+ (bmalloc::Heap::allocateSmallChunk):
+ (bmalloc::Heap::deallocateSmallChunk):
+ (bmalloc::Heap::allocateSmallPage):
+ (bmalloc::Heap::deallocateSmallLine):
+ (bmalloc::Heap::allocateSmallBumpRangesByMetadata):
+ (bmalloc::Heap::allocateSmallBumpRangesByObject):
+ (bmalloc::Heap::splitAndAllocate):
+ (bmalloc::Heap::tryAllocateLarge):
+ (bmalloc::Heap::allocateLarge):
+ (bmalloc::Heap::isLarge):
+ (bmalloc::Heap::largeSize):
+ (bmalloc::Heap::shrinkLarge):
+ (bmalloc::Heap::deallocateLarge):
+ (bmalloc::Heap::externalCommit):
+ (bmalloc::Heap::externalDecommit):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::allocateSmallBumpRanges):
+ (bmalloc::Heap::derefSmallLine):
+ * bmalloc/IsoDirectory.h:
+ * bmalloc/IsoDirectoryInlines.h:
+ (bmalloc::passedNumPages>::takeFirstEligible):
+ (bmalloc::passedNumPages>::didBecome):
+ (bmalloc::passedNumPages>::didDecommit):
+ (bmalloc::passedNumPages>::scavengePage):
+ (bmalloc::passedNumPages>::scavenge):
+ (bmalloc::passedNumPages>::scavengeToHighWatermark):
+ (bmalloc::passedNumPages>::freeableMemory): Deleted.
+ * bmalloc/IsoHeapImpl.h:
+ * bmalloc/IsoHeapImplInlines.h:
+ (bmalloc::IsoHeapImpl<Config>::takeFirstEligible):
+ (bmalloc::IsoHeapImpl<Config>::scavenge):
+ (bmalloc::IsoHeapImpl<Config>::scavengeToHighWatermark):
+ (bmalloc::IsoHeapImpl<Config>::freeableMemory):
+ (bmalloc::IsoHeapImpl<Config>::isNowFreeable):
+ (bmalloc::IsoHeapImpl<Config>::isNoLongerFreeable):
+ * bmalloc/LargeMap.cpp:
+ (bmalloc::LargeMap::remove):
+ (bmalloc::LargeMap::markAllAsEligibile):
+ * bmalloc/LargeMap.h:
+ (bmalloc::LargeMap::size):
+ (bmalloc::LargeMap::at):
+ * bmalloc/LargeRange.h:
+ (bmalloc::LargeRange::setEligible):
+ (bmalloc::LargeRange::isEligibile const):
+ (bmalloc::canMerge):
+ * bmalloc/ObjectType.cpp:
+ (bmalloc::objectType):
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::PrintTime::PrintTime):
+ (bmalloc::PrintTime::~PrintTime):
+ (bmalloc::PrintTime::print):
+ (bmalloc::Scavenger::timeSinceLastFullScavenge):
+ (bmalloc::Scavenger::timeSinceLastPartialScavenge):
+ (bmalloc::Scavenger::scavenge):
+ (bmalloc::Scavenger::partialScavenge):
+ (bmalloc::Scavenger::freeableMemory):
+ (bmalloc::Scavenger::threadRunLoop):
+ * bmalloc/Scavenger.h:
+ * bmalloc/SmallLine.h:
+ (bmalloc::SmallLine::refCount):
+ (bmalloc::SmallLine::ref):
+ (bmalloc::SmallLine::deref):
+ * bmalloc/SmallPage.h:
+ (bmalloc::SmallPage::refCount):
+ (bmalloc::SmallPage::hasFreeLines const):
+ (bmalloc::SmallPage::setHasFreeLines):
+ (bmalloc::SmallPage::ref):
+ (bmalloc::SmallPage::deref):
+ * bmalloc/bmalloc.cpp:
+ (bmalloc::api::tryLargeZeroedMemalignVirtual):
+ (bmalloc::api::freeLargeVirtual):
+
+2018-04-09 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ [bmalloc] Name Scavenger thread "bmalloc scavenger"
+ https://bugs.webkit.org/show_bug.cgi?id=166684
+
+ Reviewed by Saam Barati.
+
+ We name the thread for bmalloc Scavenger "bmalloc scavenger".
+ It is useful for debugging. In Linux environment, it will be
+ shown in GDB.
+
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::Scavenger::threadRunLoop):
+ (bmalloc::Scavenger::setName):
+ * bmalloc/Scavenger.h:
+
+2018-04-09 Michael Catanzaro <mcatanzaro@igalia.com>
+
+ Rename UNUSED to BUNUSED
+ https://bugs.webkit.org/show_bug.cgi?id=184093
+
+ Reviewed by Yusuke Suzuki.
+
+ * bmalloc/BAssert.h:
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmValidate):
+ (bmalloc::vmValidatePhysical):
+
+2018-04-08 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ Use alignas instead of compiler-specific attributes
+ https://bugs.webkit.org/show_bug.cgi?id=183508
+
+ Reviewed by Mark Lam.
+
+ Use alignas for g_gigacageBasePtr. We also add reinterpret_cast to fix
+ compile errors in ARMv7 and MIPS JSCOnly ports.
+
+ * bmalloc/Gigacage.cpp:
+ * bmalloc/Gigacage.h:
+ (Gigacage::basePtrs):
+
+2018-04-06 Saam Barati <sbarati@apple.com>
+
+ bmalloc virtual allocation API should not treat memory it vends as dirty with respect to how it drives the scavenger
+ https://bugs.webkit.org/show_bug.cgi?id=184342
+
+ Reviewed by Mark Lam.
+
+ Currently, the only user of this API is Wasm. Ideally, Wasm would tell
+ us exactly which page is dirtied. We should really do that at some point:
+ https://bugs.webkit.org/show_bug.cgi?id=184207
+
+ However, until we do that, it's better to treat none of the virtual memory
+ we vend as dirty, versus what we do now, which is treat it all as dirty.
+ This dirty memory tracking helps drive the scavenger, so on iOS, having the
+ scavenger think its under memory pressure because of memory it can't free isn't
+ useful.
+
+ * bmalloc/bmalloc.cpp:
+ (bmalloc::api::tryLargeZeroedMemalignVirtual):
+ (bmalloc::api::freeLargeVirtual):
+ * bmalloc/bmalloc.h:
+
+2018-04-05 Saam Barati <sbarati@apple.com>
+
+ IsoHeapImpl not IsoHeapImplBase should add itself to AllIsoHeaps
+ https://bugs.webkit.org/show_bug.cgi?id=184174
+
+ Reviewed by Filip Pizlo.
+
+ Otherwise, another thread may see a non-fully formed IsoHeapImpl.
+
+ * bmalloc/IsoHeapImpl.cpp:
+ (bmalloc::IsoHeapImplBase::IsoHeapImplBase):
+ (bmalloc::IsoHeapImplBase::addToAllIsoHeaps):
+ * bmalloc/IsoHeapImpl.h:
+ * bmalloc/IsoHeapImplInlines.h:
+ (bmalloc::IsoHeapImpl<Config>::IsoHeapImpl):
+
+2018-04-05 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ bmalloc StaticMutex's constructor should be constexpr
+ https://bugs.webkit.org/show_bug.cgi?id=180600
+
+ Reviewed by Mark Lam.
+
+ StaticMutex and Mutex can be unified. This patch changes std::atomic_flag in StaticMutex
+ to std::atomic<bool> to add constexpr constructor to StaticMutex. Then, StaticMutex can
+ be initialized in static storage without calling any static initializers.
+ And we also rename StaticMutex to Mutex simply.
+
+ * CMakeLists.txt:
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/AllIsoHeaps.cpp:
+ (bmalloc::AllIsoHeaps::AllIsoHeaps):
+ * bmalloc/AllIsoHeaps.h:
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::tryAllocate):
+ (bmalloc::Allocator::allocateImpl):
+ (bmalloc::Allocator::reallocate):
+ (bmalloc::Allocator::refillAllocatorSlowCase):
+ (bmalloc::Allocator::allocateLarge):
+ * bmalloc/CryptoRandom.cpp:
+ (bmalloc::ARC4RandomNumberGenerator::ARC4RandomNumberGenerator):
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::scavenge):
+ (bmalloc::Deallocator::processObjectLog):
+ (bmalloc::Deallocator::deallocateSlowCase):
+ * bmalloc/Deallocator.h:
+ (bmalloc::Deallocator::lineCache):
+ * bmalloc/DebugHeap.cpp:
+ (bmalloc::DebugHeap::DebugHeap):
+ * bmalloc/DebugHeap.h:
+ * bmalloc/Environment.cpp:
+ (bmalloc::Environment::Environment):
+ * bmalloc/Environment.h:
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::disablePrimitiveGigacage):
+ (Gigacage::addPrimitiveDisableCallback):
+ (Gigacage::removePrimitiveDisableCallback):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::freeableMemory):
+ (bmalloc::Heap::scavenge):
+ (bmalloc::Heap::deallocateLineCache):
+ (bmalloc::Heap::allocateSmallChunk):
+ (bmalloc::Heap::allocateSmallPage):
+ (bmalloc::Heap::deallocateSmallLine):
+ (bmalloc::Heap::allocateSmallBumpRangesByMetadata):
+ (bmalloc::Heap::allocateSmallBumpRangesByObject):
+ (bmalloc::Heap::splitAndAllocate):
+ (bmalloc::Heap::tryAllocateLarge):
+ (bmalloc::Heap::allocateLarge):
+ (bmalloc::Heap::isLarge):
+ (bmalloc::Heap::largeSize):
+ (bmalloc::Heap::shrinkLarge):
+ (bmalloc::Heap::deallocateLarge):
+ (bmalloc::Heap::externalCommit):
+ (bmalloc::Heap::externalDecommit):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::mutex):
+ (bmalloc::Heap::allocateSmallBumpRanges):
+ (bmalloc::Heap::derefSmallLine):
+ * bmalloc/IsoDeallocator.h:
+ * bmalloc/IsoHeap.h:
+ * bmalloc/IsoTLSDeallocatorEntry.h:
+ * bmalloc/IsoTLSDeallocatorEntryInlines.h:
+ (bmalloc::IsoTLSDeallocatorEntry<Config>::IsoTLSDeallocatorEntry):
+ * bmalloc/IsoTLSInlines.h:
+ (bmalloc::IsoTLS::ensureHeap):
+ * bmalloc/IsoTLSLayout.cpp:
+ (bmalloc::IsoTLSLayout::IsoTLSLayout):
+ (bmalloc::IsoTLSLayout::add):
+ * bmalloc/IsoTLSLayout.h:
+ * bmalloc/Mutex.cpp: Renamed from Source/bmalloc/bmalloc/StaticMutex.cpp.
+ (bmalloc::Mutex::lockSlowCase):
+ * bmalloc/Mutex.h:
+ (bmalloc::sleep):
+ (bmalloc::waitUntilFalse):
+ (bmalloc::Mutex::try_lock):
+ (bmalloc::Mutex::lock):
+ (bmalloc::Mutex::unlock):
+ (bmalloc::Mutex::Mutex): Deleted.
+ * bmalloc/ObjectType.cpp:
+ (bmalloc::objectType):
+ * bmalloc/PerProcess.cpp:
+ (bmalloc::getPerProcessData):
+ * bmalloc/PerProcess.h:
+ (bmalloc::PerProcess::mutex):
+ (bmalloc::PerProcess::getSlowCase):
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::Scavenger::Scavenger):
+ (bmalloc::Scavenger::scavenge):
+ (bmalloc::Scavenger::freeableMemory):
+ * bmalloc/Scavenger.h:
+ * bmalloc/SmallLine.h:
+ (bmalloc::SmallLine::refCount):
+ (bmalloc::SmallLine::ref):
+ (bmalloc::SmallLine::deref):
+ * bmalloc/SmallPage.h:
+ (bmalloc::SmallPage::refCount):
+ (bmalloc::SmallPage::hasFreeLines const):
+ (bmalloc::SmallPage::setHasFreeLines):
+ (bmalloc::SmallPage::ref):
+ (bmalloc::SmallPage::deref):
+ * bmalloc/StaticMutex.h: Removed.
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::VMHeap):
+ * bmalloc/VMHeap.h:
+ * bmalloc/Zone.cpp:
+ (bmalloc::Zone::Zone):
+ * bmalloc/Zone.h:
+ * bmalloc/bmalloc.cpp:
+ (bmalloc::api::tryLargeZeroedMemalignVirtual):
+ (bmalloc::api::freeLargeVirtual):
+ (bmalloc::api::isEnabled):
+ (bmalloc::api::setScavengerThreadQOSClass):
+ * bmalloc/bmalloc.h:
+
+2018-04-04 Konstantin Tokarev <annulen@yandex.ru>
+
+ Enable Gigacage unconditionally when building JSCOnly on macOS (build fix)
+ https://bugs.webkit.org/show_bug.cgi?id=184301
+
+ Reviewed by Yusuke Suzuki.
+
+ bmalloc/ProcessCheck.mm implements specific behavior for Mac and iOS ports,
+ which is guarded with BPLATFORM(COCOA). if we don't enable BPLATFORM(MAC)
+ or BPLATFORM(IOS) in JSCOnly, then BPLATFORM(COCOA) won't be defined
+ as well, and code path from ProcessCheck.mm will not be taken.
+
+ * CMakeLists.txt: Exclude ProcessCheck.mm from port-independent file
+ list.
+ * PlatformMac.cmake: Build ProcessCheck.mm for Mac port.
+ * bmalloc/BPlatform.h: Don't enable BPLATFORM(MAC) or BPLATFORM(IOS)
+ when building JSCOnly port.
+
+2018-04-03 Saam Barati <sbarati@apple.com>
+
+ totalPhysicalSize calculation when splitting a range must account for double rounding effects
+ https://bugs.webkit.org/show_bug.cgi?id=184275
+
+ Reviewed by Mark Lam.
+
+ The rounding error could happen when we split a range where the
+ range's total physical size equals the range's total size. The
+ rounding may cause the left size to lose a byte, and the right
+ size to gain a byte. This caused the right side to be a byte
+ large than its size.
+
+ * bmalloc/LargeRange.h:
+ (bmalloc::LargeRange::LargeRange):
+ (bmalloc::LargeRange::split const):
+
+2018-04-02 Saam Barati <sbarati@apple.com>
+
+ bmalloc should compute its own estimate of its footprint
+ https://bugs.webkit.org/show_bug.cgi?id=184121
+
+ Reviewed by Filip Pizlo.
+
+ This patch makes it so that bmalloc keeps track of its own physical
+ footprint.
+
+ Doing this for IsoHeaps is trivial. It allocates/deallocates fixed
+ page sizes at a time. IsoHeapImpl just updates a count every time
+ a page is committed/decommitted.
+
+ Making Heap keep its footprint was a bit trickier because of how
+ LargeRange is constructed. Before this patch, LargeRange kept track
+ of the amount of physical memory at the start of its range. This
+ patch extends large range to also keep track of the total physical memory
+ in the range just for footprint bookkeeping. This was needed to make
+ Heap's footprint come close to resembling reality, because as we merge and split
+ large ranges, the start physical size often becomes wildly inaccurate.
+ The total physical size number stored in LargeRange is still just an
+ estimate. It's possible that as ranges are split, that the total physical
+ size split amongst the two ranges doesn't resemble reality. This can
+ happen when the total physical size is really all in one end of the split,
+ but we mark it as being proportionally split amongst the resulting two
+ ranges. In practice, I did not notice this being a problem. The footprint
+ estimate tracks reality very closely (in my testing, within less than 1MB for
+ heaps with sizes upwards of 1GB). The other nice thing about total physical
+ size is that even if it diverges from reality in terms of how memory is
+ using up physical RAM, it stays internally consistent inside bmalloc's
+ own data structures.
+
+ The main oversight of this patch is how it deals with Wasm memory. All Wasm
+ memory will be viewed by bmalloc as taking up physical space even when it
+ may not be. Wasm memory starts off as taking up purely virtual pages. When a
+ page is first accessed, only then will the OS page it in and cause it to use
+ physical RAM. I opened a bug to come up with a solution to this problem:
+ https://bugs.webkit.org/show_bug.cgi?id=184207
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/AvailableMemory.cpp:
+ (bmalloc::memoryStatus):
+ * bmalloc/BPlatform.h:
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::freeableMemory):
+ (bmalloc::Heap::footprint):
+ (bmalloc::Heap::scavenge):
+ (bmalloc::Heap::deallocateSmallChunk):
+ (bmalloc::Heap::allocateSmallPage):
+ (bmalloc::Heap::splitAndAllocate):
+ (bmalloc::Heap::tryAllocateLarge):
+ (bmalloc::Heap::shrinkLarge):
+ (bmalloc::Heap::deallocateLarge):
+ (bmalloc::Heap::externalCommit):
+ (bmalloc::Heap::externalDecommit):
+ * bmalloc/Heap.h:
+ * bmalloc/IsoDirectory.h:
+ * bmalloc/IsoDirectoryInlines.h:
+ (bmalloc::passedNumPages>::takeFirstEligible):
+ (bmalloc::passedNumPages>::didDecommit):
+ (bmalloc::passedNumPages>::freeableMemory):
+ * bmalloc/IsoHeapImpl.h:
+ * bmalloc/IsoHeapImplInlines.h:
+ (bmalloc::IsoHeapImpl<Config>::freeableMemory):
+ (bmalloc::IsoHeapImpl<Config>::footprint):
+ (bmalloc::IsoHeapImpl<Config>::didCommit):
+ (bmalloc::IsoHeapImpl<Config>::didDecommit):
+ * bmalloc/LargeRange.h:
+ (bmalloc::LargeRange::LargeRange):
+ (bmalloc::LargeRange::startPhysicalSize const):
+ (bmalloc::LargeRange::setStartPhysicalSize):
+ (bmalloc::LargeRange::totalPhysicalSize const):
+ (bmalloc::LargeRange::setTotalPhysicalSize):
+ (bmalloc::merge):
+ (bmalloc::LargeRange::split const):
+ (bmalloc::LargeRange::physicalSize const): Deleted.
+ (bmalloc::LargeRange::setPhysicalSize): Deleted.
+ * bmalloc/PhysicalPageMap.h: Added.
+ This class is added for debugging purposes. It's useful when hacking
+ on the code that calculates the footprint to use this map as a sanity
+ check. It's just a simple implementation that has a set of all the committed pages.
+
+ (bmalloc::PhysicalPageMap::commit):
+ (bmalloc::PhysicalPageMap::decommit):
+ (bmalloc::PhysicalPageMap::footprint):
+ (bmalloc::PhysicalPageMap::forEachPhysicalPage):
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::dumpStats):
+ (bmalloc::Scavenger::scavenge):
+ (bmalloc::Scavenger::freeableMemory):
+ This is here just for debugging for now. But we should implement an
+ efficient version of this to use when driving when to run the
+ scavenger.
+
+ (bmalloc::Scavenger::footprint):
+ (bmalloc::Scavenger::threadRunLoop):
+ * bmalloc/Scavenger.h:
+ * bmalloc/VMAllocate.h:
+ (bmalloc::physicalPageSizeSloppy):
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::tryAllocateLargeChunk):
+ * bmalloc/bmalloc.cpp:
+ (bmalloc::api::commitAlignedPhysical):
+ (bmalloc::api::decommitAlignedPhysical):
+ * bmalloc/bmalloc.h:
+
+2018-03-28 Commit Queue <commit-queue@webkit.org>
+
+ Unreviewed, rolling out r230005.
+ https://bugs.webkit.org/show_bug.cgi?id=184115
+
+ "it caused a huge regression on iOS" (Requested by saamyjoon
+ on #webkit).
+
+ Reverted changeset:
+
+ "memoryStatus() is wrong in certain testing scenarios on iOS"
+ https://bugs.webkit.org/show_bug.cgi?id=184050
+ https://trac.webkit.org/changeset/230005
+
+2018-03-27 Saam Barati <sbarati@apple.com>
+
+ memoryStatus() is wrong in certain testing scenarios on iOS
+ https://bugs.webkit.org/show_bug.cgi?id=184050
+ <rdar://problem/37959258>
+
+ Rubber-stamped by Mark Lam.
+
+ This switches us from using "phys_footprint" to using "internal + compressed"
+ when computing the dirty memory in the current process. There are iOS testing
+ scenarios where phys_footprint doesn't give us a reliable answer. In my testing,
+ "internal + compressed" tracks phys_footprint closely (when phys_footprint is
+ working). They're usually within much less than 1% of each other. We're making
+ this change to ensure testing in our iOS infrastructure is valid.
+
+ I opened a bug to move back to phys_footprint when it's feasible:
+ https://bugs.webkit.org/show_bug.cgi?id=184050
+
+ * bmalloc/AvailableMemory.cpp:
+ (bmalloc::memoryStatus):
+
+2018-03-20 Tim Horton <timothy_horton@apple.com>
+
+ Add and adopt WK_PLATFORM_NAME and adjust default feature defines
+ https://bugs.webkit.org/show_bug.cgi?id=183758
+ <rdar://problem/38017644>
+
+ Reviewed by Dan Bernstein.
+
+ * Configurations/Base.xcconfig:
+
+2018-03-16 Filip Pizlo <fpizlo@apple.com>
+
+ Put the DOM in IsoHeaps
+ https://bugs.webkit.org/show_bug.cgi?id=183546
+
+ Reviewed by Simon Fraser.
+
+ Make it easy to runtime-disable IsoHeaps.
+
+ * bmalloc/Allocator.h:
+ * bmalloc/IsoTLS.cpp:
+ (bmalloc::IsoTLS::determineMallocFallbackState):
+ * bmalloc/IsoTLS.h:
+ * bmalloc/IsoTLSInlines.h:
+ (bmalloc::IsoTLS::allocateSlow):
+ (bmalloc::IsoTLS::deallocateSlow):
+
+2018-03-16 Michael Catanzaro <mcatanzaro@igalia.com>
+
+ Improve error message when Gigacage cannot allocate virtual memory
+ https://bugs.webkit.org/show_bug.cgi?id=183329
+
+ Reviewed by Filip Pizlo.
+
+ We've discovered that Deja Dup monitor sets a virtual memory limit, breaking Gigacage. Since
+ it runs in the background on a fresh out-of-the-box install of Ubuntu, this is not good.
+ That will have to be fixed by Deja Dup, but there is concern that other applications might
+ try this, or that users will set a virtual memory limit for the entire desktop session. Of
+ particular concern is the possibility that users might have copypasted a ulimit line into
+ a session startup script without understanding it. Let's try to make it slightly easier to
+ understand what's going wrong.
+
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::ensureGigacage):
+
+2018-03-13 Tim Horton <timothy_horton@apple.com>
+
+ Add and adopt WK_ALTERNATE_FRAMEWORKS_DIR in WTF and bmalloc
+ https://bugs.webkit.org/show_bug.cgi?id=183576
+ <rdar://problem/38396766>
+
+ Reviewed by Dan Bernstein.
+
+ * Configurations/Base.xcconfig:
+ * Configurations/bmalloc.xcconfig:
+ * Configurations/mbmalloc.xcconfig:
+
+2018-03-10 Filip Pizlo <fpizlo@apple.com>
+
+ PerProcess<> should be safe by default
+ https://bugs.webkit.org/show_bug.cgi?id=183545
+
+ Reviewed by Yusuke Suzuki.
+
+ This makes PerProcess<> safe by default, so we don't need SafePerProcess<>.
+
+ The new PerProcess<> design relies on a hash-consing mechanism for PerProcess<> storage based
+ on the __PRETTY_FUNCTION__ from inside PerProcess<>, which captures the instantiated type in
+ the string. Therefore, this can be used to runtime-coalesce PerProcess<> instances based on
+ type.
+
+ I expect this to be perf-neutral. It's an important prerequisite to more bmalloc work, since I
+ don't want to have more PerProcess<> vs SafePerProcess<> bugs, and SafePerProcess<> can't be
+ used for everything (I don't see how to use it for isoheaps).
+
+ * CMakeLists.txt:
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ * bmalloc/IsoDirectoryInlines.h:
+ (bmalloc::passedNumPages>::takeFirstEligible):
+ (bmalloc::passedNumPages>::didBecome):
+ * bmalloc/PerProcess.cpp: Added.
+ (bmalloc::stringHash):
+ (bmalloc::allocate):
+ (bmalloc::getPerProcessData):
+ * bmalloc/PerProcess.h:
+ (bmalloc::PerProcess::mutex):
+ (bmalloc::PerProcess::coalesce):
+ (bmalloc::PerProcess::getSlowCase):
+ (): Deleted.
+ * bmalloc/Scavenger.cpp:
+ * bmalloc/Scavenger.h:
+ * bmalloc/bmalloc.cpp:
+ (bmalloc::api::scavenge):
+ (bmalloc::api::setScavengerThreadQOSClass):
+
+2018-03-10 Commit Queue <commit-queue@webkit.org>
+
+ Unreviewed, rolling out r229436.
+ https://bugs.webkit.org/show_bug.cgi?id=183542
+
+ seems to have regressed wasm compile times by 10% (Requested
+ by pizlo-mbp on #webkit).
+
+ Reverted changeset:
+
+ "bmalloc mutex should be adaptive"
+ https://bugs.webkit.org/show_bug.cgi?id=177839
+ https://trac.webkit.org/changeset/229436
+
+2018-03-08 Filip Pizlo <fpizlo@apple.com>
+
+ bmalloc mutex should be adaptive
+ https://bugs.webkit.org/show_bug.cgi?id=177839
+
+ Reviewed by Michael Saboff.
+
+ This pulls the WordLock algorithm into bmalloc, mostly by copy-pasting the code. We need to
+ copy paste because sometimes we build WTF without bmalloc, so WTF cannot rely on bmalloc for
+ anything other than malloc.
+
+ Reland after failing to reproduce the WasmBench crash that caused it to get rolled out. Maybe that fixed
+ itself somehow?
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::compareExchangeWeak):
+ (bmalloc::compareExchangeStrong):
+ * bmalloc/PerThread.h:
+ * bmalloc/StaticMutex.cpp:
+ (bmalloc::StaticMutex::lockSlow):
+ (bmalloc::StaticMutex::unlockSlow):
+ (bmalloc::StaticMutex::lockSlowCase): Deleted.
+ * bmalloc/StaticMutex.h:
+ (bmalloc::StaticMutex::try_lock):
+ (bmalloc::StaticMutex::isLocked const):
+ (bmalloc::StaticMutex::init):
+ (bmalloc::StaticMutex::tryLock):
+ (bmalloc::StaticMutex::lock):
+ (bmalloc::StaticMutex::unlock):
+ (bmalloc::sleep): Deleted.
+ (bmalloc::waitUntilFalse): Deleted.
+
+2018-02-16 Filip Pizlo <fpizlo@apple.com>
+
+ Unreviewed, roll out r228306 (custom memcpy/memset) because the bots say that it was not a
+ progression.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::fastCopy): Deleted.
+ (bmalloc::fastZeroFill): Deleted.
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate):
+ * bmalloc/Bits.h:
+ (bmalloc::BitsWordOwner::operator=):
+ (bmalloc::BitsWordOwner::clearAll):
+ (bmalloc::BitsWordOwner::set):
+ * bmalloc/IsoPageInlines.h:
+ (bmalloc::IsoPage<Config>::IsoPage):
+ * bmalloc/Vector.h:
+ (bmalloc::Vector<T>::reallocateBuffer):
+
+2018-02-09 Carlos Alberto Lopez Perez <clopez@igalia.com>
+
+ Improve of string.h include after r228317.
+ https://bugs.webkit.org/show_bug.cgi?id=182642
+
+ Reviewed by Mark Lam.
+
+ * bmalloc/Algorithm.h: Avoid an architecture-specific #include.
+
+2018-02-09 Carlos Alberto Lopez Perez <clopez@igalia.com>
+
+ Fix build for !BCPU(X86_64) after r228306
+ https://bugs.webkit.org/show_bug.cgi?id=182563
+
+ Unreviewed build fix.
+
+ * bmalloc/Algorithm.h:
+
+2018-02-08 Filip Pizlo <fpizlo@apple.com>
+
+ Experiment with alternative implementation of memcpy/memset
+ https://bugs.webkit.org/show_bug.cgi?id=182563
+
+ Reviewed by Michael Saboff and Mark Lam.
+
+ Add a faster x86_64-specific implementation of memcpy and memset. Ideally, this would just be
+ implemented in WTF, but we have to copy it into bmalloc since bmalloc sits below WTF on the
+ stack.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::fastCopy):
+ (bmalloc::fastZeroFill):
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate):
+ * bmalloc/Bits.h:
+ (bmalloc::BitsWordOwner::operator=):
+ (bmalloc::BitsWordOwner::clearAll):
+ (bmalloc::BitsWordOwner::set):
+ * bmalloc/IsoPageInlines.h:
+ (bmalloc::IsoPage<Config>::IsoPage):
+ * bmalloc/Vector.h:
+ (bmalloc::Vector<T>::reallocateBuffer):
+
+2018-02-05 JF Bastien <jfbastien@apple.com>
+
+ Gigacage: enable only for WebContent process and token executables
+ https://bugs.webkit.org/show_bug.cgi?id=182457
+ <rdar://problem/35875011>
+
+ Reviewed by Keith Miller.
+
+ Gigacage is a solid security improvement, but it's probably best
+ to roll it out incrementally to the most valuable targets first
+ and progressively try out more and more over time rather than
+ outright enabling it everywhere. We've gotten some reports that it
+ has some side-effects that weren't expected, so for now let's
+ enable it for the WebContent process, JSC, and other executables
+ we know, and then later we'll enable more gigacage uses.
+
+ For now I've chosen the following bundles:
+
+ - com.apple.WebKit.WebContent.Development
+ - com.apple.WebKit.WebContent
+ - com.apple.WebProcess
+
+ And the following processes:
+
+ - jsc
+ - wasm
+ - anything starting with "test", to match the JSC tests
+
+ I tried a different approach first, where I add a function to turn
+ gigacage on or off and crash if gigacage is initialized without
+ having been told what to do. Doing this in ChildProcess and a
+ bunch of the process initialization methods isn't sufficient. I
+ got MiniBrowser working, but some other builds use static globals
+ which themselves use hash and string which are allocate with
+ bmalloc and therefore which initialize gigacage before main is
+ called and before the process gets a chance to opt in our out. It
+ gets tricky with API calls too, because we have to do the right
+ thing in any entry an API user could plausibly use, even the
+ private ones, so I endend up having to initialize gigacage in e.g.
+ WebPreferencesExperimentalFeatures.cpp.erb.
+
+ Another approach could be to create a free-for-all gigacage
+ entitlement, and opt-in the processes we want..
+
+ As a follow-up we can also check that gigacage allocation always
+ succeeds if it was allowed for that process. With my change I
+ expect it to always succeed.
+
+ * CMakeLists.txt:
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/BPlatform.h:
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::shouldBeEnabled):
+ * bmalloc/ProcessCheck.h: Added.
+ (bmalloc::gigacageEnabledForProcess):
+ * bmalloc/ProcessCheck.mm: Added.
+ (bmalloc::gigacageEnabledForProcess):
+
+2018-02-05 Joseph Pecoraro <pecoraro@apple.com>
+
+ Multiple bmalloc scavenger threads is unexpected
+ https://bugs.webkit.org/show_bug.cgi?id=182474
+ <rdar://problem/37175526>
+
+ Reviewed by Filip Pizlo.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ * bmalloc/IsoDirectoryInlines.h:
+ (bmalloc::passedNumPages>::takeFirstEligible):
+ (bmalloc::passedNumPages>::didBecome):
+ * bmalloc/bmalloc.cpp:
+ (bmalloc::api::scavenge):
+ (bmalloc::api::setScavengerThreadQOSClass):
+ Switch to SafePerProcess for Scavenger to ensure one instance
+ for the entire process.
+
+ * bmalloc/PerProcess.h:
+ (bmalloc::PerProcess::get):
+ (bmalloc::PerProcess::getFastCase):
+ (bmalloc::PerProcess::getSlowCase):
+ (bmalloc::SafePerProcess::get):
+ (bmalloc::SafePerProcess::getFastCase):
+ (bmalloc::SafePerProcess::getSlowCase):
+ Duplicate the class with a version that can ensure
+ single instances by requiring exporting symbols that
+ can be created with macros.
+
+ * bmalloc/Scavenger.cpp:
+ * bmalloc/Scavenger.h:
+ Export symbols to ensure all images get the same instance.
+
+2018-01-31 Saam Barati <sbarati@apple.com>
+
+ Replace tryLargeMemalignVirtual with tryLargeZeroedMemalignVirtual and use it to allocate large zeroed memory in Wasm
+ https://bugs.webkit.org/show_bug.cgi?id=182064
+ <rdar://problem/36840132>
+
+ Reviewed by Geoffrey Garen.
+
+ This patch replaces the tryLargeMemalignVirtual API with tryLargeZeroedMemalignVirtual.
+ By doing that, we're able to remove the AllocationKind enum. To zero the memory,
+ tryLargeZeroedMemalignVirtual uses mmap(... MAP_ANON ...) over previously mmapped
+ memory. This both purges the any resident memory for the virtual range and ensures
+ that the pages in the range are zeroed. Most OSs should implement this by taking a
+ page fault and zero filling on first access. Therefore, this API is returning pages
+ that will result in page faults on first access. Hence, the name 'virtual' in the API.
+ This API differs from the old API in that users of it need not call madvise themselves.
+ The memory is ready to go.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/AllocationKind.h: Removed.
+ * bmalloc/DebugHeap.cpp:
+ (bmalloc::DebugHeap::memalignLarge):
+ (bmalloc::DebugHeap::freeLarge):
+ * bmalloc/DebugHeap.h:
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::splitAndAllocate):
+ (bmalloc::Heap::tryAllocateLarge):
+ (bmalloc::Heap::allocateLarge):
+ (bmalloc::Heap::shrinkLarge):
+ (bmalloc::Heap::deallocateLarge):
+ * bmalloc/Heap.h:
+ * bmalloc/IsoPage.cpp:
+ (bmalloc::IsoPageBase::allocatePageMemory):
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmZeroAndPurge):
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::tryAllocateLargeChunk):
+ * bmalloc/VMHeap.h:
+ * bmalloc/bmalloc.cpp:
+ (bmalloc::api::tryLargeZeroedMemalignVirtual):
+ (bmalloc::api::freeLargeVirtual):
+ (bmalloc::api::tryLargeMemalignVirtual): Deleted.
+ * bmalloc/bmalloc.h:
+
+2018-01-19 Keith Miller <keith_miller@apple.com>
+
+ HaveInternalSDK includes should be "#include?"
+ https://bugs.webkit.org/show_bug.cgi?id=179670
+
+ Reviewed by Dan Bernstein.
+
+ * Configurations/Base.xcconfig:
+
+2018-01-18 Dan Bernstein <mitz@apple.com>
+
+ [Xcode] Streamline and future-proof target-macOS-version-dependent build setting definitions
+ https://bugs.webkit.org/show_bug.cgi?id=181803
+
+ Reviewed by Tim Horton.
+
+ * Configurations/Base.xcconfig: Updated.
+ * Configurations/DebugRelease.xcconfig: Ditto.
+
+2018-01-16 Michael Catanzaro <mcatanzaro@igalia.com>
+
+ mbmalloc should only be built in developer mode
+ https://bugs.webkit.org/show_bug.cgi?id=181654
+
+ Reviewed by Carlos Garcia Campos.
+
+ * CMakeLists.txt:
+
+2018-01-15 Michael Catanzaro <mcatanzaro@igalia.com>
+
+ Improve use of ExportMacros
+ https://bugs.webkit.org/show_bug.cgi?id=181652
+
+ Reviewed by Konstantin Tokarev.
+
+ Disable BEXPORT on Linux ports.
+
+ * bmalloc/BExport.h: Check for BUSE(EXPORT_MACROS).
+ * bmalloc/BPlatform.h: Add BUSE(EXPORT_MACROS) and define it on macOS and iOS.
+
+2017-12-20 Ting-Wei Lan <lantw44@gmail.com>
+
+ Include stdio.h before using stderr and _IONBF
+ https://bugs.webkit.org/show_bug.cgi?id=181046
+
+ Reviewed by Alex Christensen.
+
+ * bmalloc/IsoTLS.cpp:
+
+2017-12-14 David Kilzer <ddkilzer@apple.com>
+
+ Enable -Wstrict-prototypes for WebKit
+ <https://webkit.org/b/180757>
+ <rdar://problem/36024132>
+
+ Rubber-stamped by Joseph Pecoraro.
+
+ * Configurations/Base.xcconfig:
+ (CLANG_WARN_STRICT_PROTOTYPES): Add. Set to YES.
+
+2017-12-14 Saam Barati <sbarati@apple.com>
+
+ logVMFailure should not simulate crash on iOS
+ https://bugs.webkit.org/show_bug.cgi?id=180790
+
+ Reviewed by JF Bastien.
+
+ The Gigacage allocation on iOS is expected to fail in certain circumstances.
+ Let's not simulate a crash on failure because since this is expected behavior.
+
+ * bmalloc/VMAllocate.h:
+ (bmalloc::tryVMAllocate):
+
+2017-12-11 Tim Horton <timothy_horton@apple.com>
+
+ Stop using deprecated target conditional for simulator builds
+ https://bugs.webkit.org/show_bug.cgi?id=180662
+ <rdar://problem/35136156>
+
+ Reviewed by Simon Fraser.
+
+ * bmalloc/BPlatform.h:
+
+2017-12-08 Saam Barati <sbarati@apple.com>
+
+ Enable gigacage on iOS with a 32GB runway and ensure it doesn't break WasmBench
+ https://bugs.webkit.org/show_bug.cgi?id=178557
+
+ Reviewed by Mark Lam.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::isPowerOfTwo):
+ * bmalloc/Gigacage.cpp:
+ * bmalloc/Gigacage.h:
+
+2017-12-05 Andy Estes <aestes@apple.com>
+
+ [Darwin] Simplify use of TargetConditionals
+ https://bugs.webkit.org/show_bug.cgi?id=180455
+ <rdar://problem/35142971>
+
+ Reviewed by Tim Horton.
+
+ There's no need to check if TARGET_* macros are defined on Darwin platforms, since
+ TargetConditionals.h always defines them. Also, we can simplify
+ (TARGET_OS_EMBEDDED || TARGET_OS_IPHONE || TARGET_IPHONE_SIMULATOR) to TARGET_OS_IPHONE.
+
+ * bmalloc/BPlatform.h:
+
+2017-12-05 Filip Pizlo <fpizlo@apple.com>
+
+ bmalloc IsoHeap needs to allow a thread to deallocate some size for the first time
+ https://bugs.webkit.org/show_bug.cgi?id=180443
+
+ Reviewed by Saam Barati.
+
+ It's true that we can expect a heap to already be initialized if we try to deallocate in it. But it
+ may not have its deallocator initialized on this thread yet.
+
+ This is easily fixed by adding a null check on the deallocate path. That's probably not going to
+ change perf at all. But doing that allows me to get rid of a lot of weird stuff I previously did to
+ avoid that null check, like creating a dummy TLS in the DebugHeap case.
+
+ * bmalloc/IsoTLS.cpp:
+ (bmalloc::IsoTLS::debugFree):
+ (bmalloc::IsoTLS::deallocateSlow): Deleted.
+ * bmalloc/IsoTLS.h:
+ * bmalloc/IsoTLSInlines.h:
+ (bmalloc::IsoTLS::allocateImpl):
+ (bmalloc::IsoTLS::allocateSlow):
+ (bmalloc::IsoTLS::deallocateImpl):
+ (bmalloc::IsoTLS::deallocateSlow):
+ (bmalloc::IsoTLS::ensureHeapAndEntries):
+
+2017-12-01 Michael Saboff <msaboff@apple.com>
+
+ Gigacage should not be enabled for ARM64_32
+ https://bugs.webkit.org/show_bug.cgi?id=180265
+
+ Reviewed by Saam Barati.
+
+ Disabled Gigacage for ARM64_32.
+ In the process, restructured Gigacage::shouldBeEnabled() with GIGACAGE_ENABLED set
+ to 0 to avoid a dead code compiler warning.
+
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::shouldBeEnabled):
+ * bmalloc/Gigacage.h:
+
+2017-11-29 JF Bastien <jfbastien@apple.com>
+
+ WTF / bmalloc: don't write to 0xbbadbeef when ASAN is looking
+ https://bugs.webkit.org/show_bug.cgi?id=180175
+
+ Reviewed by Mark Lam.
+
+ ASAN knows that 0xbbadbeef is a bbad aaddress, and tells us so
+ when we write to it, say in an assert. That creates bbad error
+ reports where ASAN thinks we write to an invalid address, instead
+ of thinking that we hit an assertion. In some cases, tooling that
+ use fuzzers aggregate similar issues, and think that we just have
+ the one bug and not a bunch of different asserts.
+
+ At the same time, bmalloc's version of CRASH just writes to
+ 0xbbadbeef and assumes that's invalid and will crash, which isn't
+ necessarily true on non-Mac platforms. WTF's version then makes
+ sure there's a crash, so bmalloc should do the same.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/BAssert.h:
+ * bmalloc/BCompiler.h: Added.
+ * bmalloc/BPlatform.h:
+
+2017-11-27 Filip Pizlo <fpizlo@apple.com>
+
+ Don't crash in forEachEntry when DebugHeap is enabled.
+
+ Unreviewed, fixing crashes on leaks bots by removing an assertion.
+
+ * bmalloc/IsoTLS.cpp:
+ (bmalloc::IsoTLS::forEachEntry):
+ * test/testbmalloc.cpp: Make this test work with DebugHeap so I can catch this kind of problem in the future.
+
+2017-11-16 Filip Pizlo <fpizlo@apple.com>
+
+ Isolated Heaps caused an increase in reported leaks on the bots
+ https://bugs.webkit.org/show_bug.cgi?id=179463
+
+ Reviewed by Darin Adler.
+
+ This fixes the way isoheaps interact with system tools:
+
+ - Opts into the VMHeap API so that the leaks tool can find isoheap memory.
+
+ - Opts into the DebugHeap/Environment APIs so that we turn off isoheap allocation if memory
+ debugging options are in use.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/DebugHeap.h:
+ * bmalloc/IsoHeap.h:
+ * bmalloc/IsoPage.cpp: Added.
+ (bmalloc::IsoPageBase::allocatePageMemory):
+ * bmalloc/IsoPage.h:
+ * bmalloc/IsoPageInlines.h:
+ (bmalloc::IsoPage<Config>::tryCreate):
+ * bmalloc/IsoTLS.cpp:
+ (bmalloc::IsoTLS::deallocateSlow):
+ (bmalloc::IsoTLS::ensureEntries):
+ (bmalloc::IsoTLS::isUsingDebugHeap):
+ (bmalloc::IsoTLS::debugMalloc):
+ * bmalloc/IsoTLS.h:
+ * bmalloc/IsoTLSInlines.h:
+ (bmalloc::IsoTLS::allocate):
+ (bmalloc::IsoTLS::deallocate):
+ (bmalloc::IsoTLS::allocateImpl):
+ (bmalloc::IsoTLS::allocateFast):
+ (bmalloc::IsoTLS::allocateSlow):
+ (bmalloc::IsoTLS::deallocateImpl):
+ (bmalloc::IsoTLS::deallocateFast):
+ (bmalloc::IsoTLS::ensureHeapAndEntries):
+ (bmalloc::IsoTLS::allocator): Deleted.
+ (bmalloc::IsoTLS::deallocator): Deleted.
+ * bmalloc/bmalloc.cpp:
+ (bmalloc::api::tryLargeMemalignVirtual):
+ (bmalloc::api::freeLargeVirtual):
+ (bmalloc::api::scavenge):
+ (bmalloc::api::isEnabled):
+ (bmalloc::api::setScavengerThreadQOSClass):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::tryLargeMemalignVirtual): Deleted.
+ (bmalloc::api::freeLargeVirtual): Deleted.
+ (bmalloc::api::scavenge): Deleted.
+ (bmalloc::api::isEnabled): Deleted.
+ (bmalloc::api::setScavengerThreadQOSClass): Deleted.
+
+2017-11-14 Saam Barati <sbarati@apple.com>
+
+ Make the gigacage runway 32GB
+ https://bugs.webkit.org/show_bug.cgi?id=175062
+
+ Reviewed by Mark Lam.
+
+ Making the gigacage runway 32GB defends us against buffer overflows in the
+ cage reaching memory outside the cage assuming indices are 32-bit unsigned
+ integers and the type they're indexing into has size <= 8 bytes. This is
+ exactly the case for many things in JSC. For example, butterfly access in
+ JSC meet this criteria, as does typed array access.
+
+ The 32GB comes from 8 * 2^32 = 32GB.
+
+ * bmalloc/Gigacage.cpp:
+
+2017-11-08 Michael Catanzaro <mcatanzaro@igalia.com>
+
+ Gigacage.cpp:44:46: warning: ‘*’ in boolean context, suggest ‘&&’ instead [-Wint-in-bool-context]
+ https://bugs.webkit.org/show_bug.cgi?id=179427
+
+ Reviewed by Saam Barati.
+
+ Tweak the conditional to suppress the warning.
+
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::ensureGigacage):
+
+2017-11-07 Saam Barati <sbarati@apple.com>
+
+ We should PROT_NONE the Gigacage runway so OOB accesses crash
+ https://bugs.webkit.org/show_bug.cgi?id=179392
+
+ Reviewed by Mark Lam.
+
+ If we assume that an attacker will exploit JSC and cause OOB accesses,
+ we should make OOB accesses in the Gigacage runway crash.
+
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::ensureGigacage):
+
+2017-10-31 Filip Pizlo <fpizlo@apple.com>
+
+ bmalloc should support strictly type-segregated isolated heaps
+ https://bugs.webkit.org/show_bug.cgi?id=178108
+
+ Reviewed by Saam Barati, Simon Fraser, and Ryosuke Niwa.
+
+ This introduces a new allocation API in bmalloc called IsoHeap. An IsoHeap is templatized by
+ type and created in static storage. When unused, it takes only a few words. When you do use
+ it, each IsoHeap gets a bag of virtual pages unique to it. This prevents use-after-free bugs
+ in one IsoHeap from affecting any other memory. At worst, two pointers of the same type will
+ point to the same object even though they should not have.
+
+ IsoHeaps allocate using a first-fit discipline that combines ideas from bmalloc and Riptide
+ (the JSC GC):
+
+ Like Riptide, it uses a bump'n'pop allocator. What Riptide calls blocks, IsoHeaps calls
+ pages. Pages are collected into directories. Directories track pages using bitvectors, so
+ that it's easy to quickly find a completely free page or one that has at least one free
+ object. I think that the bump'n'pop allocator is as fast as the bmalloc Immix-style (page and
+ line) allocator, but is better at allocating in holes. It's guaranteed to follow a first-fit
+ discipline. However, the real reason why I wrote it that was is that this is what I'm more
+ familiar with. This is a part of the design I want to revisit (bug 179278).
+
+ Like bmalloc, it uses a deallocation log. This means that the internal IsoHeap data
+ structures can be locked with a coarse-grained lock, since the deallocator only grabs it when
+ flushing the log. Similarly, the allocator only grabs it when refilling the bump'n'pop
+ FreeList.
+
+ This adds a unit test for IsoHeaps. In this change, IsoHeaps are adopted only by WebCore's
+ RenderObject.
+
+ Note that despite the use of GC concepts, it's not a goal to make this code directly sharable
+ with GC. The GC will probably have to do isolated heaps its own way (likely a special
+ Subspace or something like that).
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Algorithm.h:
+ (bmalloc::findBitInWord):
+ * bmalloc/AllIsoHeaps.cpp: Added.
+ (bmalloc::AllIsoHeaps::AllIsoHeaps):
+ (bmalloc::AllIsoHeaps::add):
+ (bmalloc::AllIsoHeaps::head):
+ * bmalloc/AllIsoHeaps.h: Added.
+ * bmalloc/AllIsoHeapsInlines.h: Added.
+ (bmalloc::AllIsoHeaps::forEach):
+ * bmalloc/BMalloced.h: Added.
+ * bmalloc/Bits.h: Added.
+ (bmalloc::bitsArrayLength):
+ (bmalloc::BitsWordView::BitsWordView):
+ (bmalloc::BitsWordView::numBits const):
+ (bmalloc::BitsWordView::word const):
+ (bmalloc::BitsWordOwner::BitsWordOwner):
+ (bmalloc::BitsWordOwner::view const):
+ (bmalloc::BitsWordOwner::operator=):
+ (bmalloc::BitsWordOwner::setAll):
+ (bmalloc::BitsWordOwner::clearAll):
+ (bmalloc::BitsWordOwner::set):
+ (bmalloc::BitsWordOwner::numBits const):
+ (bmalloc::BitsWordOwner::arrayLength const):
+ (bmalloc::BitsWordOwner::word const):
+ (bmalloc::BitsWordOwner::word):
+ (bmalloc::BitsWordOwner::words const):
+ (bmalloc::BitsWordOwner::words):
+ (bmalloc::BitsAndWords::BitsAndWords):
+ (bmalloc::BitsAndWords::view const):
+ (bmalloc::BitsAndWords::numBits const):
+ (bmalloc::BitsAndWords::word const):
+ (bmalloc::BitsOrWords::BitsOrWords):
+ (bmalloc::BitsOrWords::view const):
+ (bmalloc::BitsOrWords::numBits const):
+ (bmalloc::BitsOrWords::word const):
+ (bmalloc::BitsNotWords::BitsNotWords):
+ (bmalloc::BitsNotWords::view const):
+ (bmalloc::BitsNotWords::numBits const):
+ (bmalloc::BitsNotWords::word const):
+ (bmalloc::BitsImpl::BitsImpl):
+ (bmalloc::BitsImpl::numBits const):
+ (bmalloc::BitsImpl::size const):
+ (bmalloc::BitsImpl::arrayLength const):
+ (bmalloc::BitsImpl::operator== const):
+ (bmalloc::BitsImpl::operator!= const):
+ (bmalloc::BitsImpl::at const):
+ (bmalloc::BitsImpl::operator[] const):
+ (bmalloc::BitsImpl::isEmpty const):
+ (bmalloc::BitsImpl::operator& const):
+ (bmalloc::BitsImpl::operator| const):
+ (bmalloc::BitsImpl::operator~ const):
+ (bmalloc::BitsImpl::forEachSetBit const):
+ (bmalloc::BitsImpl::forEachClearBit const):
+ (bmalloc::BitsImpl::forEachBit const):
+ (bmalloc::BitsImpl::findBit const):
+ (bmalloc::BitsImpl::findSetBit const):
+ (bmalloc::BitsImpl::findClearBit const):
+ (bmalloc::BitsImpl::wordView const):
+ (bmalloc::BitsImpl::atImpl const):
+ (bmalloc::Bits::Bits):
+ (bmalloc::Bits::operator=):
+ (bmalloc::Bits::resize):
+ (bmalloc::Bits::setAll):
+ (bmalloc::Bits::clearAll):
+ (bmalloc::Bits::setAndCheck):
+ (bmalloc::Bits::operator|=):
+ (bmalloc::Bits::operator&=):
+ (bmalloc::Bits::at const):
+ (bmalloc::Bits::operator[] const):
+ (bmalloc::Bits::BitReference::BitReference):
+ (bmalloc::Bits::BitReference::operator bool const):
+ (bmalloc::Bits::BitReference::operator=):
+ (bmalloc::Bits::at):
+ (bmalloc::Bits::operator[]):
+ * bmalloc/CryptoRandom.cpp: Replaced with Source/bmalloc/bmalloc/CryptoRandom.cpp.
+ (bmalloc::cryptoRandom):
+ * bmalloc/CryptoRandom.h: Replaced with Source/bmalloc/bmalloc/CryptoRandom.h.
+ * bmalloc/DeferredDecommit.h: Added.
+ * bmalloc/DeferredDecommitInlines.h: Added.
+ (bmalloc::DeferredDecommit::DeferredDecommit):
+ * bmalloc/DeferredTrigger.h: Added.
+ (bmalloc::DeferredTrigger::DeferredTrigger):
+ * bmalloc/DeferredTriggerInlines.h: Added.
+ (bmalloc::DeferredTrigger<trigger>::didBecome):
+ (bmalloc::DeferredTrigger<trigger>::handleDeferral):
+ * bmalloc/EligibilityResult.h: Added.
+ (bmalloc::EligibilityResult::EligibilityResult):
+ * bmalloc/EligibilityResultInlines.h: Added.
+ (bmalloc::EligibilityResult<Config>::EligibilityResult):
+ * bmalloc/FixedVector.h:
+ * bmalloc/FreeList.cpp: Added.
+ (bmalloc::FreeList::FreeList):
+ (bmalloc::FreeList::~FreeList):
+ (bmalloc::FreeList::clear):
+ (bmalloc::FreeList::initializeList):
+ (bmalloc::FreeList::initializeBump):
+ (bmalloc::FreeList::contains const):
+ * bmalloc/FreeList.h: Added.
+ (bmalloc::FreeCell::scramble):
+ (bmalloc::FreeCell::descramble):
+ (bmalloc::FreeCell::setNext):
+ (bmalloc::FreeCell::next const):
+ (bmalloc::FreeList::allocationWillFail const):
+ (bmalloc::FreeList::allocationWillSucceed const):
+ (bmalloc::FreeList::originalSize const):
+ (bmalloc::FreeList::head const):
+ * bmalloc/FreeListInlines.h: Added.
+ (bmalloc::FreeList::allocate):
+ (bmalloc::FreeList::forEach const):
+ * bmalloc/IsoAllocator.h: Added.
+ * bmalloc/IsoAllocatorInlines.h: Added.
+ (bmalloc::IsoAllocator<Config>::IsoAllocator):
+ (bmalloc::IsoAllocator<Config>::~IsoAllocator):
+ (bmalloc::IsoAllocator<Config>::allocate):
+ (bmalloc::IsoAllocator<Config>::allocateSlow):
+ (bmalloc::IsoAllocator<Config>::scavenge):
+ * bmalloc/IsoConfig.h: Added.
+ * bmalloc/IsoDeallocator.h: Added.
+ * bmalloc/IsoDeallocatorInlines.h: Added.
+ (bmalloc::IsoDeallocator<Config>::IsoDeallocator):
+ (bmalloc::IsoDeallocator<Config>::~IsoDeallocator):
+ (bmalloc::IsoDeallocator<Config>::deallocate):
+ (bmalloc::IsoDeallocator<Config>::scavenge):
+ * bmalloc/IsoDirectory.h: Added.
+ (bmalloc::IsoDirectoryBaseBase::IsoDirectoryBaseBase):
+ (bmalloc::IsoDirectoryBaseBase::~IsoDirectoryBaseBase):
+ (bmalloc::IsoDirectoryBase::heap):
+ * bmalloc/IsoDirectoryInlines.h: Added.
+ (bmalloc::IsoDirectoryBase<Config>::IsoDirectoryBase):
+ (bmalloc::passedNumPages>::IsoDirectory):
+ (bmalloc::passedNumPages>::takeFirstEligible):
+ (bmalloc::passedNumPages>::didBecome):
+ (bmalloc::passedNumPages>::didDecommit):
+ (bmalloc::passedNumPages>::scavenge):
+ (bmalloc::passedNumPages>::forEachCommittedPage):
+ * bmalloc/IsoDirectoryPage.h: Added.
+ (bmalloc::IsoDirectoryPage::index const):
+ * bmalloc/IsoDirectoryPageInlines.h: Added.
+ (bmalloc::IsoDirectoryPage<Config>::IsoDirectoryPage):
+ (bmalloc::IsoDirectoryPage<Config>::pageFor):
+ * bmalloc/IsoHeap.h: Added.
+ (bmalloc::api::IsoHeap::allocatorOffset):
+ (bmalloc::api::IsoHeap::setAllocatorOffset):
+ (bmalloc::api::IsoHeap::deallocatorOffset):
+ (bmalloc::api::IsoHeap::setDeallocatorOffset):
+ * bmalloc/IsoHeapImpl.cpp: Added.
+ (bmalloc::IsoHeapImplBase::IsoHeapImplBase):
+ (bmalloc::IsoHeapImplBase::~IsoHeapImplBase):
+ (bmalloc::IsoHeapImplBase::scavengeNow):
+ (bmalloc::IsoHeapImplBase::finishScavenging):
+ * bmalloc/IsoHeapImpl.h: Added.
+ * bmalloc/IsoHeapImplInlines.h: Added.
+ (bmalloc::IsoHeapImpl<Config>::IsoHeapImpl):
+ (bmalloc::IsoHeapImpl<Config>::takeFirstEligible):
+ (bmalloc::IsoHeapImpl<Config>::didBecomeEligible):
+ (bmalloc::IsoHeapImpl<Config>::scavenge):
+ (bmalloc::IsoHeapImpl<Config>::allocatorOffset):
+ (bmalloc::IsoHeapImpl<Config>::deallocatorOffset):
+ (bmalloc::IsoHeapImpl<Config>::numLiveObjects):
+ (bmalloc::IsoHeapImpl<Config>::numCommittedPages):
+ (bmalloc::IsoHeapImpl<Config>::forEachDirectory):
+ (bmalloc::IsoHeapImpl<Config>::forEachCommittedPage):
+ (bmalloc::IsoHeapImpl<Config>::forEachLiveObject):
+ * bmalloc/IsoHeapInlines.h: Added.
+ (bmalloc::api::IsoHeap<Type>::allocate):
+ (bmalloc::api::IsoHeap<Type>::tryAllocate):
+ (bmalloc::api::IsoHeap<Type>::deallocate):
+ (bmalloc::api::IsoHeap<Type>::scavenge):
+ (bmalloc::api::IsoHeap<Type>::isInitialized):
+ (bmalloc::api::IsoHeap<Type>::impl):
+ * bmalloc/IsoPage.h: Added.
+ (bmalloc::IsoPage::index const):
+ (bmalloc::IsoPage::directory):
+ (bmalloc::IsoPage::isInUseForAllocation const):
+ (bmalloc::IsoPage::indexOfFirstObject):
+ * bmalloc/IsoPageInlines.h: Added.
+ (bmalloc::IsoPage<Config>::tryCreate):
+ (bmalloc::IsoPage<Config>::IsoPage):
+ (bmalloc::IsoPage<Config>::free):
+ (bmalloc::IsoPage<Config>::startAllocating):
+ (bmalloc::IsoPage<Config>::stopAllocating):
+ (bmalloc::IsoPage<Config>::forEachLiveObject):
+ * bmalloc/IsoPageTrigger.h: Added.
+ * bmalloc/IsoTLS.cpp: Added.
+ (bmalloc::IsoTLS::scavenge):
+ (bmalloc::IsoTLS::IsoTLS):
+ (bmalloc::IsoTLS::ensureEntries):
+ (bmalloc::IsoTLS::destructor):
+ (bmalloc::IsoTLS::sizeForCapacity):
+ (bmalloc::IsoTLS::capacityForSize):
+ (bmalloc::IsoTLS::size):
+ (bmalloc::IsoTLS::forEachEntry):
+ * bmalloc/IsoTLS.h: Added.
+ * bmalloc/IsoTLSAllocatorEntry.h: Added.
+ * bmalloc/IsoTLSAllocatorEntryInlines.h: Added.
+ (bmalloc::IsoTLSAllocatorEntry<Config>::IsoTLSAllocatorEntry):
+ (bmalloc::IsoTLSAllocatorEntry<Config>::~IsoTLSAllocatorEntry):
+ (bmalloc::IsoTLSAllocatorEntry<Config>::construct):
+ * bmalloc/IsoTLSDeallocatorEntry.h: Added.
+ * bmalloc/IsoTLSDeallocatorEntryInlines.h: Added.
+ (bmalloc::IsoTLSDeallocatorEntry<Config>::IsoTLSDeallocatorEntry):
+ (bmalloc::IsoTLSDeallocatorEntry<Config>::~IsoTLSDeallocatorEntry):
+ (bmalloc::IsoTLSDeallocatorEntry<Config>::construct):
+ * bmalloc/IsoTLSEntry.cpp: Added.
+ (bmalloc::IsoTLSEntry::IsoTLSEntry):
+ (bmalloc::IsoTLSEntry::~IsoTLSEntry):
+ * bmalloc/IsoTLSEntry.h: Added.
+ (bmalloc::IsoTLSEntry::offset const):
+ (bmalloc::IsoTLSEntry::alignment const):
+ (bmalloc::IsoTLSEntry::size const):
+ (bmalloc::IsoTLSEntry::extent const):
+ * bmalloc/IsoTLSEntryInlines.h: Added.
+ (bmalloc::IsoTLSEntry::walkUpToInclusive):
+ (bmalloc::DefaultIsoTLSEntry<EntryType>::DefaultIsoTLSEntry):
+ (bmalloc::DefaultIsoTLSEntry<EntryType>::~DefaultIsoTLSEntry):
+ (bmalloc::DefaultIsoTLSEntry<EntryType>::move):
+ (bmalloc::DefaultIsoTLSEntry<EntryType>::destruct):
+ (bmalloc::DefaultIsoTLSEntry<EntryType>::scavenge):
+ * bmalloc/IsoTLSInlines.h: Added.
+ (bmalloc::IsoTLS::allocate):
+ (bmalloc::IsoTLS::deallocate):
+ (bmalloc::IsoTLS::scavenge):
+ (bmalloc::IsoTLS::allocator):
+ (bmalloc::IsoTLS::deallocator):
+ (bmalloc::IsoTLS::get):
+ (bmalloc::IsoTLS::set):
+ (bmalloc::IsoTLS::ensureHeap):
+ (bmalloc::IsoTLS::ensureHeapAndEntries):
+ * bmalloc/IsoTLSLayout.cpp: Added.
+ (bmalloc::IsoTLSLayout::IsoTLSLayout):
+ (bmalloc::IsoTLSLayout::add):
+ * bmalloc/IsoTLSLayout.h: Added.
+ (bmalloc::IsoTLSLayout::head const):
+ * bmalloc/PerHeapKind.h:
+ * bmalloc/PerProcess.h:
+ (bmalloc::PerProcess<T>::getFastCase):
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::Scavenger::scavenge):
+ * bmalloc/Scavenger.h:
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::scavengeThisThread):
+ * test: Added.
+ * test/testbmalloc.cpp: Added.
+ (hiddenTruthBecauseNoReturnIsStupid):
+ (usage):
+ (assertEmptyPointerSet):
+ (assertHasObjects):
+ (assertHasOnlyObjects):
+ (assertClean):
+ (testIsoSimple):
+ (testIsoSimpleScavengeBeforeDealloc):
+ (testIsoFlipFlopFragmentedPages):
+ (testIsoFlipFlopFragmentedPagesScavengeInMiddle):
+ (BisoMalloced::BisoMalloced):
+ (testBisoMalloced):
+ (BisoMallocedInline::BisoMallocedInline):
+ (testBisoMallocedInline):
+ (run):
+ (main):
+
+2017-10-30 Zan Dobersek <zdobersek@igalia.com>
+
+ [ARM64][Linux] Re-enable Gigacage
+ https://bugs.webkit.org/show_bug.cgi?id=178130
+
+ Reviewed by Michael Catanzaro.
+
+ * bmalloc/Gigacage.h: Re-enable Gigacage on ARM64 Linux.
+
+2017-10-25 Commit Queue <commit-queue@webkit.org>
+
+ Unreviewed, rolling out r222945.
+ https://bugs.webkit.org/show_bug.cgi?id=178818
+
+ "It made WasmBench crash" (Requested by saamyjoon on #webkit).
+
+ Reverted changeset:
+
+ "bmalloc mutex should be adaptive"
+ https://bugs.webkit.org/show_bug.cgi?id=177839
+ https://trac.webkit.org/changeset/222945
+
+2017-10-24 Zan Dobersek <zdobersek@igalia.com>
+
+ [Linux] Enable Gigacage in x64 Linux environment
+ https://bugs.webkit.org/show_bug.cgi?id=177745
+ <rdar://problem/34773148>
+
+ Reviewed by Yusuke Suzuki.
+
+ Re-enable Gigacage on x86_64 Linux platforms after it was disabled in 223877.
+
+ The cause for the revert was problems with huge coredumps being generated
+ while Gigacage was enabled. The feature virtually allocates about 80GB of
+ memory at the beginning of the process lifetime. This is not a problem in
+ itself since the memory range is marked as not needed through madvise(),
+ but all this memory was still included upon core dump generation on Linux.
+ Since there are reasonable limits enforced upon core dumps, these were
+ being truncated every time, not yielding any useful information.
+
+ To avoid this, on Linux, invocations of madvise() with the MADV_NORMAL and
+ MADV_DONTNEED advice parameters should be accompanied with respectively
+ matching MADV_DODUMP and MADV_DONTDUMP madvise() calls. This correctly
+ avoids core-dumping any memory that's not yet been physically allocated.
+
+ * bmalloc/Gigacage.h:
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmDeallocatePhysicalPages):
+ (bmalloc::vmAllocatePhysicalPages):
+
+2017-10-24 David Kilzer <ddkilzer@apple.com>
+
+ Need to pass non-nil argument to SimulateCrash() in bmalloc::logVMFailure()
+ <https://webkit.org/b/178740>
+ <rdar://problem/35154943>
+
+ Reviewed by Saam Barati.
+
+ * bmalloc/BPlatform.h:
+ (BUNUSED_PARAM): Define macro.
+ * bmalloc/Logging.cpp:
+ (SimulateCrash): Change third argument of SimulateCrash() to
+ CFStringRef since it's an NSString * in Objective-C.
+ (bmalloc::logVMFailure): Create a CFStringRef to use as a
+ description string. Use new vmSize parameter to log size.
+ * bmalloc/Logging.h:
+ (bmalloc::logVMFailure): Update function signature to take a
+ size_t parameter representing vmSize.
+ * bmalloc/VMAllocate.h:
+ (bmalloc::tryVMAllocate): Pass vmSize into logVMFailure().
+
+2017-10-23 Michael Catanzaro <mcatanzaro@igalia.com>
+
+ Unreviewed, roll out r222731
+ https://bugs.webkit.org/show_bug.cgi?id=177745
+ <rdar://problem/34773148>
+
+ Unfortunately Gigacage has broken core dump generation.
+
+ * bmalloc/Gigacage.h:
+
+2017-10-23 Zan Dobersek <zdobersek@igalia.com>
+
+ bmalloc::api::tryLargeMemalignVirtual() shouldn't assert on a failed allocation
+ https://bugs.webkit.org/show_bug.cgi?id=178654
+
+ Reviewed by Geoffrey Garen.
+
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::tryLargeMemalignVirtual): Call Heap::tryAllocateLarge()
+ instead of Heap::allocateLarge(). The former will return a null pointer
+ upon a failed allocation, allowing the caller to fail gracefully just as
+ the API entrypoint implies, while the latter currently provokes a crash
+ in these circumstances.
+
+2017-10-19 Saam Barati <sbarati@apple.com>
+
+ Runtime disable gigacage on iOS because it broke WasmBench
+ https://bugs.webkit.org/show_bug.cgi?id=178556
+
+ Reviewed by Keith Miller.
+
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::shouldBeEnabled):
+
+2017-10-17 Filip Pizlo <fpizlo@apple.com>
+
+ You can't vmDeallocate null
+ <rdar://problem/35038926>
+
+ Reviewed by Michael Saboff.
+
+ After failing allocation, we would try to deallocate the thing we failed to allocate. The fix is to
+ not try to deallocate something that is obviously null.
+
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::ensureGigacage):
+
+2017-09-29 Filip Pizlo <fpizlo@apple.com>
+
+ Enable gigacage on iOS
+ https://bugs.webkit.org/show_bug.cgi?id=177586
+
+ Reviewed by JF Bastien.
+
+ Introduce the ability to disable gigacage at runtime if allocation fails. If any step of gigacage
+ allocation fails, we free all of the gigacages and turn off gigacage support.
+
+ Roll this back in after discussion.
+
+ * CMakeLists.txt:
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Cache.cpp:
+ (bmalloc::Cache::scavenge):
+ * bmalloc/Cache.h:
+ (bmalloc::Cache::tryAllocate):
+ (bmalloc::Cache::allocate):
+ (bmalloc::Cache::deallocate):
+ (bmalloc::Cache::reallocate):
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::ensureGigacage):
+ (Gigacage::runway):
+ (Gigacage::totalSize):
+ (Gigacage::shouldBeEnabled):
+ (): Deleted.
+ (Gigacage::Callback::Callback): Deleted.
+ (Gigacage::Callback::function): Deleted.
+ (Gigacage::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks): Deleted.
+ * bmalloc/Gigacage.h:
+ (Gigacage::wasEnabled):
+ (Gigacage::isEnabled):
+ (Gigacage::runway): Deleted.
+ (Gigacage::totalSize): Deleted.
+ * bmalloc/HeapKind.cpp: Added.
+ (bmalloc::isActiveHeapKind):
+ (bmalloc::mapToActiveHeapKind):
+ * bmalloc/HeapKind.h:
+ (bmalloc::isActiveHeapKindAfterEnsuringGigacage):
+ (bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::Scavenger::scavenge):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::tryLargeMemalignVirtual):
+ (bmalloc::api::freeLargeVirtual):
+ (bmalloc::api::isEnabled):
+
+2017-10-11 Commit Queue <commit-queue@webkit.org>
+
+ Unreviewed, rolling out r223113 and r223121.
+ https://bugs.webkit.org/show_bug.cgi?id=178182
+
+ Reintroduced 20% regression on Kraken (Requested by rniwa on
+ #webkit).
+
+ Reverted changesets:
+
+ "Enable gigacage on iOS"
+ https://bugs.webkit.org/show_bug.cgi?id=177586
+ https://trac.webkit.org/changeset/223113
+
+ "Use one virtual allocation for all gigacages and their
+ runways"
+ https://bugs.webkit.org/show_bug.cgi?id=178050
+ https://trac.webkit.org/changeset/223121
+
+2017-10-07 Filip Pizlo <fpizlo@apple.com>
+
+ Use one virtual allocation for all gigacages and their runways
+ https://bugs.webkit.org/show_bug.cgi?id=178050
+
+ Reviewed by Saam Barati.
+
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::ensureGigacage):
+ (Gigacage::runway): Deleted.
+ (Gigacage::totalSize): Deleted.
+ * bmalloc/Gigacage.h:
+
+2017-09-29 Filip Pizlo <fpizlo@apple.com>
+
+ Enable gigacage on iOS
+ https://bugs.webkit.org/show_bug.cgi?id=177586
+
+ Reviewed by JF Bastien.
+
+ Introduce the ability to disable gigacage at runtime if allocation fails. If any step of gigacage
+ allocation fails, we free all of the gigacages and turn off gigacage support.
+
+ Reland this after confirming that the 20% Kraken regression was a one-bot fluke. Local testing on the
+ same kind of system did not show the regression. Saam and I both tried independently.
+
+ * CMakeLists.txt:
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Cache.cpp:
+ (bmalloc::Cache::scavenge):
+ * bmalloc/Cache.h:
+ (bmalloc::Cache::tryAllocate):
+ (bmalloc::Cache::allocate):
+ (bmalloc::Cache::deallocate):
+ (bmalloc::Cache::reallocate):
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::ensureGigacage):
+ (Gigacage::runway):
+ (Gigacage::totalSize):
+ (Gigacage::shouldBeEnabled):
+ (): Deleted.
+ (Gigacage::Callback::Callback): Deleted.
+ (Gigacage::Callback::function): Deleted.
+ (Gigacage::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks): Deleted.
+ * bmalloc/Gigacage.h:
+ (Gigacage::wasEnabled):
+ (Gigacage::isEnabled):
+ (Gigacage::runway): Deleted.
+ (Gigacage::totalSize): Deleted.
+ * bmalloc/HeapKind.cpp: Added.
+ (bmalloc::isActiveHeapKind):
+ (bmalloc::mapToActiveHeapKind):
+ * bmalloc/HeapKind.h:
+ (bmalloc::isActiveHeapKindAfterEnsuringGigacage):
+ (bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::Scavenger::scavenge):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::tryLargeMemalignVirtual):
+ (bmalloc::api::freeLargeVirtual):
+ (bmalloc::api::isEnabled):
+
+2017-10-09 Commit Queue <commit-queue@webkit.org>
+
+ Unreviewed, rolling out r223015 and r223025.
+ https://bugs.webkit.org/show_bug.cgi?id=178093
+
+ Regressed Kraken on iOS by 20% (Requested by keith_mi_ on
+ #webkit).
+
+ Reverted changesets:
+
+ "Enable gigacage on iOS"
+ https://bugs.webkit.org/show_bug.cgi?id=177586
+ http://trac.webkit.org/changeset/223015
+
+ "Unreviewed, disable Gigacage on ARM64 Linux"
+ https://bugs.webkit.org/show_bug.cgi?id=177586
+ http://trac.webkit.org/changeset/223025
+
+2017-10-07 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ Unreviewed, disable Gigacage on ARM64 Linux
+ https://bugs.webkit.org/show_bug.cgi?id=177586
+
+ Gigacage's LLInt change breaks ARM64 Linux.
+ Currently we do not have maintainers for this.
+ Let's simply disable it.
+
+ * bmalloc/Gigacage.h:
+
+2017-09-29 Filip Pizlo <fpizlo@apple.com>
+
+ Enable gigacage on iOS
+ https://bugs.webkit.org/show_bug.cgi?id=177586
+
+ Reviewed by JF Bastien.
+
+ Introduce the ability to disable gigacage at runtime if allocation fails. If any step of gigacage
+ allocation fails, we free all of the gigacages and turn off gigacage support.
+
+ * CMakeLists.txt:
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Cache.cpp:
+ (bmalloc::Cache::scavenge):
+ * bmalloc/Cache.h:
+ (bmalloc::Cache::tryAllocate):
+ (bmalloc::Cache::allocate):
+ (bmalloc::Cache::deallocate):
+ (bmalloc::Cache::reallocate):
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::ensureGigacage):
+ (Gigacage::runway):
+ (Gigacage::totalSize):
+ (Gigacage::shouldBeEnabled):
+ (): Deleted.
+ (Gigacage::Callback::Callback): Deleted.
+ (Gigacage::Callback::function): Deleted.
+ (Gigacage::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks): Deleted.
+ * bmalloc/Gigacage.h:
+ (Gigacage::wasEnabled):
+ (Gigacage::isEnabled):
+ (Gigacage::runway): Deleted.
+ (Gigacage::totalSize): Deleted.
+ * bmalloc/HeapKind.cpp: Added.
+ (bmalloc::isActiveHeapKind):
+ (bmalloc::mapToActiveHeapKind):
+ * bmalloc/HeapKind.h:
+ (bmalloc::isActiveHeapKindAfterEnsuringGigacage):
+ (bmalloc::mapToActiveHeapKindAfterEnsuringGigacage):
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::Scavenger::scavenge):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::tryLargeMemalignVirtual):
+ (bmalloc::api::freeLargeVirtual):
+ (bmalloc::api::isEnabled):
+
+2017-10-05 Filip Pizlo <fpizlo@apple.com>
+
+ Use one Scavenger thread for all Heaps
+ https://bugs.webkit.org/show_bug.cgi?id=174973
+
+ Reviewed by JF Bastien.
+
+ This combines the scavengers from all Heap instances into a single scavenger. It also combines
+ the accounting for deciding when to run. Each Heap still controls what it means to scavenge
+ itself (it's all in Heap::scavenge) but the policy decisions are all controlled by Scavenger.
+ Because Scavenger is also the only thing that needs an AsyncTask, this removes AsyncTask and
+ moves all of AsyncTask's logic into Scavenger.
+
+ This appears to be a 1% progression on JetStream (with high statistical confidence: p = 0.0049).
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/AsyncTask.h: Removed.
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::allocateSmallChunk):
+ (bmalloc::Heap::allocateSmallPage):
+ (bmalloc::Heap::deallocateSmallLine):
+ (bmalloc::Heap::splitAndAllocate):
+ (bmalloc::Heap::tryAllocateLarge):
+ (bmalloc::Heap::shrinkLarge):
+ (bmalloc::Heap::deallocateLarge):
+ (bmalloc::Heap::concurrentScavenge): Deleted.
+ (bmalloc::Heap::scheduleScavengerIfUnderMemoryPressure): Deleted.
+ (bmalloc::Heap::scheduleScavenger): Deleted.
+ * bmalloc/Heap.h:
+ * bmalloc/Scavenger.cpp:
+ (bmalloc::Scavenger::Scavenger):
+ (bmalloc::Scavenger::run):
+ (bmalloc::Scavenger::runHoldingLock):
+ (bmalloc::Scavenger::runSoon):
+ (bmalloc::Scavenger::runSoonHoldingLock):
+ (bmalloc::Scavenger::didStartGrowing):
+ (bmalloc::Scavenger::scheduleIfUnderMemoryPressure):
+ (bmalloc::Scavenger::scheduleIfUnderMemoryPressureHoldingLock):
+ (bmalloc::Scavenger::schedule):
+ (bmalloc::Scavenger::threadEntryPoint):
+ (bmalloc::Scavenger::threadRunLoop):
+ (bmalloc::Scavenger::setSelfQOSClass):
+ * bmalloc/Scavenger.h:
+ (bmalloc::Scavenger::willRun):
+ (bmalloc::Scavenger::willRunSoon):
+
+2017-10-04 Filip Pizlo <fpizlo@apple.com>
+
+ bmalloc mutex should be adaptive
+ https://bugs.webkit.org/show_bug.cgi?id=177839
+
+ Reviewed by Michael Saboff.
+
+ This pulls the WordLock algorithm into bmalloc, mostly by copy-pasting the code. We need to
+ copy paste because sometimes we build WTF without bmalloc, so WTF cannot rely on bmalloc for
+ anything other than malloc.
+
+ Reland after fixing ancient WordLock bug: the notify_one has to happen with the lock held
+ to ensure it doesn't run after that thread has died.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::compareExchangeWeak):
+ (bmalloc::compareExchangeStrong):
+ * bmalloc/PerThread.h:
+ * bmalloc/StaticMutex.cpp:
+ (bmalloc::StaticMutex::lockSlow):
+ (bmalloc::StaticMutex::unlockSlow):
+ (bmalloc::StaticMutex::lockSlowCase): Deleted.
+ * bmalloc/StaticMutex.h:
+ (bmalloc::StaticMutex::try_lock):
+ (bmalloc::StaticMutex::isLocked const):
+ (bmalloc::StaticMutex::init):
+ (bmalloc::StaticMutex::tryLock):
+ (bmalloc::StaticMutex::lock):
+ (bmalloc::StaticMutex::unlock):
+ (bmalloc::sleep): Deleted.
+ (bmalloc::waitUntilFalse): Deleted.
+
+2017-10-05 Matt Lewis <jlewis3@apple.com>
+
+ Unreviewed, rolling out r222893.
+
+ This caused multiple API failures.
+
+ Reverted changeset:
+
+ "bmalloc mutex should be adaptive"
+ https://bugs.webkit.org/show_bug.cgi?id=177839
+ http://trac.webkit.org/changeset/222893
+
+2017-10-05 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ [Linux] Port MallocBench
+ https://bugs.webkit.org/show_bug.cgi?id=177856
+
+ Reviewed by Filip Pizlo.
+
+ * CMakeLists.txt:
+
+2017-10-04 Filip Pizlo <fpizlo@apple.com>
+
+ bmalloc mutex should be adaptive
+ https://bugs.webkit.org/show_bug.cgi?id=177839
+
+ Reviewed by Michael Saboff.
+
+ This pulls the WordLock algorithm into bmalloc, mostly by copy-pasting the code. We need to
+ copy paste because sometimes we build WTF without bmalloc, so WTF cannot rely on bmalloc for
+ anything other than malloc.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::compareExchangeWeak):
+ (bmalloc::compareExchangeStrong):
+ * bmalloc/PerThread.h:
+ * bmalloc/StaticMutex.cpp:
+ (bmalloc::StaticMutex::lockSlow):
+ (bmalloc::StaticMutex::unlockSlow):
+ (bmalloc::StaticMutex::lockSlowCase): Deleted.
+ * bmalloc/StaticMutex.h:
+ (bmalloc::StaticMutex::try_lock):
+ (bmalloc::StaticMutex::isLocked const):
+ (bmalloc::StaticMutex::init):
+ (bmalloc::StaticMutex::tryLock):
+ (bmalloc::StaticMutex::lock):
+ (bmalloc::StaticMutex::unlock):
+ (bmalloc::sleep): Deleted.
+ (bmalloc::waitUntilFalse): Deleted.
+
+2017-10-02 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ [Linux] Enable Gigacage in x64 Linux environment
+ https://bugs.webkit.org/show_bug.cgi?id=177745
+
+ Reviewed by Carlos Garcia Campos.
+
+ This patch enables Gigacage in x64 Linux environment.
+ Gigacage enforces a caged pointer to reference to the
+ specific memory region. This reduces the effectiveness
+ of some types of attacks setting a pointer to ArrayBuffer
+ and modifying arbitrary memory region.
+
+ * bmalloc/Gigacage.h:
+
+2017-09-29 Commit Queue <commit-queue@webkit.org>
+
+ Unreviewed, rolling out r222625.
+ https://bugs.webkit.org/show_bug.cgi?id=177664
+
+ causes crashes on iOS (Requested by pizlo-mbp on #webkit).
+
+ Reverted changeset:
+
+ "Enable gigacage on iOS"
+ https://bugs.webkit.org/show_bug.cgi?id=177586
+ http://trac.webkit.org/changeset/222625
+
+2017-09-28 Filip Pizlo <fpizlo@apple.com>
+
+ Enable gigacage on iOS
+ https://bugs.webkit.org/show_bug.cgi?id=177586
+
+ Reviewed by Michael Saboff.
+
+ This enables Gigacage on iOS using a much smaller cage size. It's not necessary for it to be so
+ small, but this is a good conservative starting point to start to exercise the code.
+
+ * bmalloc/Gigacage.h:
+
+2017-09-26 Filip Pizlo <fpizlo@apple.com>
+
+ Put g_gigacageBasePtr into its own page and make it read-only
+ https://bugs.webkit.org/show_bug.cgi?id=174972
+
+ Reviewed by Michael Saboff.
+
+ This puts the gigacage base pointers into their own page and makes that page read-only.
+
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::ensureGigacage):
+ (Gigacage::disablePrimitiveGigacage):
+ (Gigacage::addPrimitiveDisableCallback):
+ * bmalloc/Gigacage.h:
+ (Gigacage::basePtr):
+ (Gigacage::basePtrs):
+
+2017-09-04 Adrian Perez de Castro <aperez@igalia.com>
+
+ Unreviewed build fix for Clang with libc++
+
+ Fixes a build failure when building with Clang, -stdlib=libc++, and gigacage
+ support enabled, which resulted in "stderr" being undefined.
+
+ * bmalloc/Gigacage.cpp: Add missing <ctsdio> include to pull the definition.
+
+2017-09-03 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ Large virtual memory region allocation requires MMAP_NORESERVE in Linux
+ https://bugs.webkit.org/show_bug.cgi?id=176211
+
+ Reviewed by Geoffrey Garen.
+
+ In Linux, we cannot allocate very large memory region without MMAP_NORESERVE.
+ Linux kernel needs to reserve swap area for allocated memory region. If the
+ swap area is exhausted, kernel fails to allocate the memory region with ENOMEM.
+
+ This patch adds MMAP_NORESERVE to mmap flags in Linux. By adding this flag,
+ mmap does not need to reserve swap area for the reserved memory region.
+ This allows us to reserve very large memory region that is necessary for Gigacage.
+
+ * bmalloc/BPlatform.h:
+ * bmalloc/VMAllocate.h:
+ (bmalloc::tryVMAllocate):
+
+2017-08-22 Filip Pizlo <fpizlo@apple.com>
+
+ Strings need to be in some kind of gigacage
+ https://bugs.webkit.org/show_bug.cgi?id=174924
+
+ Reviewed by Oliver Hunt.
+
+ This adds a StringGigacage.
+
+ * bmalloc/Gigacage.cpp:
+ * bmalloc/Gigacage.h:
+ (Gigacage::name):
+ (Gigacage::basePtr):
+ (Gigacage::forEachKind):
+ * bmalloc/HeapKind.h:
+ (bmalloc::isGigacage):
+ (bmalloc::gigacageKind):
+ (bmalloc::heapKind):
+
+2017-08-30 Matt Lewis <jlewis3@apple.com>
+
+ Unreviewed, rolling out r221384.
+
+ This patch caused multiple 32-bit JSC test failures.
+
+ Reverted changeset:
+
+ "Strings need to be in some kind of gigacage"
+ https://bugs.webkit.org/show_bug.cgi?id=174924
+ http://trac.webkit.org/changeset/221384
+
+2017-08-22 Filip Pizlo <fpizlo@apple.com>
+
+ Strings need to be in some kind of gigacage
+ https://bugs.webkit.org/show_bug.cgi?id=174924
+
+ Reviewed by Oliver Hunt.
+
+ This adds a StringGigacage.
+
+ * bmalloc/Gigacage.cpp:
+ * bmalloc/Gigacage.h:
+ (Gigacage::name):
+ (Gigacage::basePtr):
+ (Gigacage::forEachKind):
+ * bmalloc/HeapKind.h:
+ (bmalloc::isGigacage):
+ (bmalloc::gigacageKind):
+ (bmalloc::heapKind):
+
+2017-08-25 Daniel Bates <dabates@apple.com>
+
+ Demarcate code added due to lack of NSDMI for aggregates
+ https://bugs.webkit.org/show_bug.cgi?id=175990
+
+ Reviewed by Andy Estes.
+
+ * bmalloc/BPlatform.h:
+ * bmalloc/List.h: Be explicit when initializing m_node to improve readability.
+ (bmalloc::ListNode::ListNode):
+
+2017-08-23 Filip Pizlo <fpizlo@apple.com>
+
+ Reduce Gigacage sizes
+ https://bugs.webkit.org/show_bug.cgi?id=175920
+
+ Reviewed by Mark Lam.
+
+ This introduces the ability to have different gigacage sizes for different gigacages, and uses it to reduce the size of both
+ gigacages, but to different extents: Primitive gets 32GB with a 16GB runway and JSValue gets 16GB.
+
+ This is a ~10% membuster progression on my Mac Pro.
+
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::ensureGigacage):
+ * bmalloc/Gigacage.h:
+ (Gigacage::size):
+ (Gigacage::alignment):
+ (Gigacage::mask):
+ (Gigacage::runway):
+ (Gigacage::totalSize):
+ (Gigacage::caged):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::gigacageSize):
+ * bmalloc/Heap.h:
+
+2017-08-08 Filip Pizlo <fpizlo@apple.com>
+
+ Baseline JIT should do caging
+ https://bugs.webkit.org/show_bug.cgi?id=175037
+
+ Reviewed by Mark Lam.
+
+ This centralizes the notion of permanently enabling the primitive gigacage, which we only do in jsc
+ and WebProcess.
+
+ This saves the baseline JIT from emitting some code. Otherwise it would always have to emit enabled
+ checks on each typed array access.
+
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::primitiveGigacageDisabled):
+ (Gigacage::disableDisablingPrimitiveGigacageIfShouldBeEnabled):
+ (Gigacage::isDisablingPrimitiveGigacageDisabled):
+ * bmalloc/Gigacage.h:
+ (Gigacage::isPrimitiveGigacagePermanentlyEnabled):
+ (Gigacage::canPrimitiveGigacageBeDisabled):
+
+2017-08-08 Ryan Haddad <ryanhaddad@apple.com>
+
+ Unreviewed, rolling out r220368.
+
+ This change caused WK1 tests to exit early with crashes.
+
+ Reverted changeset:
+
+ "Baseline JIT should do caging"
+ https://bugs.webkit.org/show_bug.cgi?id=175037
+ http://trac.webkit.org/changeset/220368
+
+2017-08-07 Filip Pizlo <fpizlo@apple.com>
+
+ Baseline JIT should do caging
+ https://bugs.webkit.org/show_bug.cgi?id=175037
+
+ Reviewed by Mark Lam.
+
+ This centralizes the notion of permanently enabling the primitive gigacage, which we only do in jsc
+ and WebProcess.
+
+ This saves the baseline JIT from emitting some code. Otherwise it would always have to emit enabled
+ checks on each typed array access.
+
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::primitiveGigacageDisabled):
+ (Gigacage::disableDisablingPrimitiveGigacageIfShouldBeEnabled):
+ (Gigacage::isDisablingPrimitiveGigacageDisabled):
+ * bmalloc/Gigacage.h:
+ (Gigacage::isPrimitiveGigacagePermanentlyEnabled):
+ (Gigacage::canPrimitiveGigacageBeDisabled):
+
+2017-08-06 Filip Pizlo <fpizlo@apple.com>
+
+ Primitive auxiliaries and JSValue auxiliaries should have separate gigacages
+ https://bugs.webkit.org/show_bug.cgi?id=174919
+
+ Reviewed by Keith Miller.
+
+ This introduces two kinds of Gigacage, Primitive and JSValue. This translates to two kinds of
+ HeapKind, PrimitiveGigacage and JSValueGigacage.
+
+ The new support functionality required turning Inline.h into BInline.h, and INLINE into BINLINE, and
+ NO_INLINE into BNO_INLINE.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::refillAllocatorSlowCase):
+ (bmalloc::Allocator::refillAllocator):
+ (bmalloc::Allocator::allocateLarge):
+ (bmalloc::Allocator::allocateLogSizeClass):
+ * bmalloc/AsyncTask.h:
+ * bmalloc/BInline.h: Copied from Source/bmalloc/bmalloc/Inline.h.
+ * bmalloc/Cache.cpp:
+ (bmalloc::Cache::tryAllocateSlowCaseNullCache):
+ (bmalloc::Cache::allocateSlowCaseNullCache):
+ (bmalloc::Cache::deallocateSlowCaseNullCache):
+ (bmalloc::Cache::reallocateSlowCaseNullCache):
+ * bmalloc/Deallocator.cpp:
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::PrimitiveDisableCallbacks::PrimitiveDisableCallbacks):
+ (Gigacage::ensureGigacage):
+ (Gigacage::disablePrimitiveGigacage):
+ (Gigacage::addPrimitiveDisableCallback):
+ (Gigacage::removePrimitiveDisableCallback):
+ (Gigacage::Callbacks::Callbacks): Deleted.
+ (Gigacage::disableGigacage): Deleted.
+ (Gigacage::addDisableCallback): Deleted.
+ (Gigacage::removeDisableCallback): Deleted.
+ * bmalloc/Gigacage.h:
+ (Gigacage::name):
+ (Gigacage::basePtr):
+ (Gigacage::forEachKind):
+ (Gigacage::caged):
+ (Gigacage::isCaged):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::usingGigacage):
+ (bmalloc::Heap::gigacageBasePtr):
+ * bmalloc/Heap.h:
+ * bmalloc/HeapKind.h:
+ (bmalloc::isGigacage):
+ (bmalloc::gigacageKind):
+ (bmalloc::heapKind):
+ * bmalloc/Inline.h: Removed.
+ * bmalloc/Map.h:
+ * bmalloc/PerProcess.h:
+ (bmalloc::PerProcess<T>::getFastCase):
+ (bmalloc::PerProcess<T>::get):
+ (bmalloc::PerProcess<T>::getSlowCase):
+ * bmalloc/PerThread.h:
+ (bmalloc::PerThread<T>::getFastCase):
+ * bmalloc/Vector.h:
+ (bmalloc::Vector<T>::push):
+ (bmalloc::Vector<T>::shrinkCapacity):
+ (bmalloc::Vector<T>::growCapacity):
+
+2017-08-02 Filip Pizlo <fpizlo@apple.com>
+
+ If Gigacage is disabled, bmalloc should service large aligned memory allocation requests through vmAllocate
+ https://bugs.webkit.org/show_bug.cgi?id=175085
+
+ Reviewed by Saam Barati.
+
+ This fixes a problem where if we used gmalloc, WebAssembly memory allocations would still use
+ bmalloc's large allocator.
+
+ We want to use the page allocator for those "large" allocations when the Gigacage is disabled.
+
+ * bmalloc/DebugHeap.cpp:
+ (bmalloc::DebugHeap::DebugHeap):
+ (bmalloc::DebugHeap::memalignLarge):
+ (bmalloc::DebugHeap::freeLarge):
+ * bmalloc/DebugHeap.h:
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::tryAllocateLarge):
+ (bmalloc::Heap::deallocateLarge):
+
+2017-08-02 Filip Pizlo <fpizlo@apple.com>
+
+ We should be OK with the gigacage being disabled on gmalloc
+ https://bugs.webkit.org/show_bug.cgi?id=175082
+
+ Reviewed by Michael Saboff.
+
+ This adds Gigacage::shouldBeEnabled(), which returns false when we're using gmalloc or other things
+ that enable DebugHeap.
+
+ * bmalloc/Environment.cpp:
+ (bmalloc::Environment::Environment):
+ * bmalloc/Environment.h:
+ * bmalloc/Gigacage.cpp:
+ (Gigacage::ensureGigacage):
+ (Gigacage::shouldBeEnabled):
+ * bmalloc/Gigacage.h:
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ * bmalloc/Heap.h:
+
+2017-08-01 Filip Pizlo <fpizlo@apple.com>
+
+ Bmalloc and GC should put auxiliaries (butterflies, typed array backing stores) in a gigacage (separate multi-GB VM region)
+ https://bugs.webkit.org/show_bug.cgi?id=174727
+
+ Reviewed by Mark Lam.
+
+ This adds a mechanism for managing multiple isolated heaps in bmalloc. For now, these isoheaps
+ (isolated heaps) have a very simple relationship with each other and with the rest of bmalloc:
+
+ - You have to choose how many isoheaps you will have statically. See numHeaps in HeapKind.h.
+
+ - Because numHeaps is static, each isoheap gets fast thread-local allocation. Basically, we have a
+ Cache for each heap kind.
+
+ - Each isoheap gets its own Heap.
+
+ - Each Heap gets a scavenger thread.
+
+ - Some things, like Zone/VMHeap/Scavenger, are per-process.
+
+ Most of the per-HeapKind functionality is handled by PerHeapKind<>.
+
+ This approach is ideal for supporting special per-HeapKind behaviors. For now we have two heaps:
+ the Primary heap for normal malloc and the Gigacage. The gigacage is a 64GB-aligned 64GB virtual
+ region that we now use for variable-length random-access allocations. No Primary allocations will
+ go into the Gigacage.
+
+ * CMakeLists.txt:
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/AllocationKind.h: Added.
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::Allocator):
+ (bmalloc::Allocator::tryAllocate):
+ (bmalloc::Allocator::allocateImpl):
+ (bmalloc::Allocator::reallocate):
+ (bmalloc::Allocator::refillAllocatorSlowCase):
+ (bmalloc::Allocator::allocateLarge):
+ * bmalloc/Allocator.h:
+ * bmalloc/BExport.h: Added.
+ * bmalloc/Cache.cpp:
+ (bmalloc::Cache::scavenge):
+ (bmalloc::Cache::Cache):
+ (bmalloc::Cache::tryAllocateSlowCaseNullCache):
+ (bmalloc::Cache::allocateSlowCaseNullCache):
+ (bmalloc::Cache::deallocateSlowCaseNullCache):
+ (bmalloc::Cache::reallocateSlowCaseNullCache):
+ (bmalloc::Cache::operator new): Deleted.
+ (bmalloc::Cache::operator delete): Deleted.
+ * bmalloc/Cache.h:
+ (bmalloc::Cache::tryAllocate):
+ (bmalloc::Cache::allocate):
+ (bmalloc::Cache::deallocate):
+ (bmalloc::Cache::reallocate):
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::Deallocator):
+ (bmalloc::Deallocator::scavenge):
+ (bmalloc::Deallocator::processObjectLog):
+ (bmalloc::Deallocator::deallocateSlowCase):
+ * bmalloc/Deallocator.h:
+ * bmalloc/Gigacage.cpp: Added.
+ (Gigacage::Callback::Callback):
+ (Gigacage::Callback::function):
+ (Gigacage::Callbacks::Callbacks):
+ (Gigacage::ensureGigacage):
+ (Gigacage::disableGigacage):
+ (Gigacage::addDisableCallback):
+ (Gigacage::removeDisableCallback):
+ * bmalloc/Gigacage.h: Added.
+ (Gigacage::caged):
+ (Gigacage::isCaged):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::usingGigacage):
+ (bmalloc::Heap::concurrentScavenge):
+ (bmalloc::Heap::splitAndAllocate):
+ (bmalloc::Heap::tryAllocateLarge):
+ (bmalloc::Heap::allocateLarge):
+ (bmalloc::Heap::shrinkLarge):
+ (bmalloc::Heap::deallocateLarge):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::mutex):
+ (bmalloc::Heap::kind const):
+ (bmalloc::Heap::setScavengerThreadQOSClass): Deleted.
+ * bmalloc/HeapKind.h: Added.
+ * bmalloc/ObjectType.cpp:
+ (bmalloc::objectType):
+ * bmalloc/ObjectType.h:
+ * bmalloc/PerHeapKind.h: Added.
+ (bmalloc::PerHeapKindBase::PerHeapKindBase):
+ (bmalloc::PerHeapKindBase::size):
+ (bmalloc::PerHeapKindBase::at):
+ (bmalloc::PerHeapKindBase::at const):
+ (bmalloc::PerHeapKindBase::operator[]):
+ (bmalloc::PerHeapKindBase::operator[] const):
+ (bmalloc::StaticPerHeapKind::StaticPerHeapKind):
+ (bmalloc::PerHeapKind::PerHeapKind):
+ (bmalloc::PerHeapKind::~PerHeapKind):
+ * bmalloc/PerThread.h:
+ (bmalloc::PerThread<T>::destructor):
+ (bmalloc::PerThread<T>::getSlowCase):
+ (bmalloc::PerThreadStorage<Cache>::get): Deleted.
+ (bmalloc::PerThreadStorage<Cache>::init): Deleted.
+ * bmalloc/Scavenger.cpp: Added.
+ (bmalloc::Scavenger::Scavenger):
+ (bmalloc::Scavenger::scavenge):
+ * bmalloc/Scavenger.h: Added.
+ (bmalloc::Scavenger::setScavengerThreadQOSClass):
+ (bmalloc::Scavenger::requestedScavengerThreadQOSClass const):
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::VMHeap):
+ (bmalloc::VMHeap::tryAllocateLargeChunk):
+ * bmalloc/VMHeap.h:
+ * bmalloc/Zone.cpp:
+ (bmalloc::Zone::Zone):
+ * bmalloc/Zone.h:
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::tryMalloc):
+ (bmalloc::api::malloc):
+ (bmalloc::api::tryMemalign):
+ (bmalloc::api::memalign):
+ (bmalloc::api::realloc):
+ (bmalloc::api::tryLargeMemalignVirtual):
+ (bmalloc::api::free):
+ (bmalloc::api::freeLargeVirtual):
+ (bmalloc::api::scavengeThisThread):
+ (bmalloc::api::scavenge):
+ (bmalloc::api::isEnabled):
+ (bmalloc::api::setScavengerThreadQOSClass):
+ * bmalloc/mbmalloc.cpp:
+
+2017-08-01 Daewoong Jang <daewoong.jang@navercorp.com>
+
+ Implement __builtin_clzl for MSVC
+ https://bugs.webkit.org/show_bug.cgi?id=174232
+
+ Reviewed by Geoffrey Garen.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::clzl):
+ (bmalloc::clzl<1>):
+ (bmalloc::__builtin_clzl):
+ * bmalloc/BPlatform.h:
+
+2017-07-31 Mark Lam <mark.lam@apple.com>
+
+ Fixed some comment typos.
+
+ Not reviewed.
+
+ * bmalloc/PerProcess.h:
+
+2017-07-14 Filip Pizlo <fpizlo@apple.com>
+
+ It should be easy to decide how WebKit yields
+ https://bugs.webkit.org/show_bug.cgi?id=174298
+
+ Reviewed by Saam Barati.
+
+ Use sched_yield() explicitly.
+
+ * bmalloc/StaticMutex.cpp:
+ (bmalloc::StaticMutex::lockSlowCase):
+
+2017-07-20 Chris Dumez <cdumez@apple.com>
+
+ Replace calls to Vector::resize() with calls to more efficient shrink() / grow() when applicable
+ https://bugs.webkit.org/show_bug.cgi?id=174660
+
+ Reviewed by Geoffrey Garen.
+
+ Replace calls to Vector::resize() with calls to more efficient shrink() / grow() when applicable.
+ This essentially replaces a branch to figure out if the new size is less or greater than the
+ current size by an assertion.
+
+ * bmalloc/Map.h:
+ (bmalloc::Hash>::rehash):
+
+2017-07-18 Andy Estes <aestes@apple.com>
+
+ [Xcode] Enable CLANG_WARN_RANGE_LOOP_ANALYSIS
+ https://bugs.webkit.org/show_bug.cgi?id=174631
+
+ Reviewed by Tim Horton.
+
+ * Configurations/Base.xcconfig:
+
+2017-07-18 Andy Estes <aestes@apple.com>
+
+ [Xcode] Enable CLANG_WARN_OBJC_LITERAL_CONVERSION
+ https://bugs.webkit.org/show_bug.cgi?id=174631
+
+ Reviewed by Sam Weinig.
+
+ * Configurations/Base.xcconfig:
+
+2017-07-18 Andy Estes <aestes@apple.com>
+
+ [Xcode] Enable CLANG_WARN_NON_LITERAL_NULL_CONVERSION
+ https://bugs.webkit.org/show_bug.cgi?id=174631
+
+ Reviewed by Dan Bernstein.
+
+ * Configurations/Base.xcconfig:
+
+2017-07-18 Andy Estes <aestes@apple.com>
+
+ [Xcode] Enable CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING
+ https://bugs.webkit.org/show_bug.cgi?id=174631
+
+ Reviewed by Darin Adler.
+
+ * Configurations/Base.xcconfig:
+
+2017-07-12 Adrian Perez de Castro <aperez@igalia.com>
+
+ bmalloc: Failure to build when the compiler specifically targets ARMv8-A / defines __ARM_ARCH_8A__
+ https://bugs.webkit.org/show_bug.cgi?id=174424
+
+ Reviewed by Michael Catanzaro.
+
+ * bmalloc/BPlatform.h: Also check for __ARCH_ARM_8A__ to detect ARMv8.
+
+2017-07-05 Daewoong Jang <daewoong.jang@navercorp.com>
+
+ reinterpret_cast does not evaluate to constexpr
+ https://bugs.webkit.org/show_bug.cgi?id=173622
+
+ Reviewed by Yusuke Suzuki.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::mask):
+ (bmalloc::roundUpToMultipleOf):
+
+2017-07-03 Andy Estes <aestes@apple.com>
+
+ [Xcode] Add an experimental setting to build with ccache
+ https://bugs.webkit.org/show_bug.cgi?id=173875
+
+ Reviewed by Tim Horton.
+
+ * Configurations/DebugRelease.xcconfig: Included ccache.xcconfig.
+
+2017-07-01 Dan Bernstein <mitz@apple.com>
+
+ [iOS] Remove code only needed when building for iOS 9.x
+ https://bugs.webkit.org/show_bug.cgi?id=174068
+
+ Reviewed by Tim Horton.
+
+ * bmalloc/BPlatform.h:
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmPageSizePhysical):
+
+2017-07-01 Dan Bernstein <mitz@apple.com>
+
+ [macOS] Remove code only needed when building for OS X Yosemite
+ https://bugs.webkit.org/show_bug.cgi?id=174067
+
+ Reviewed by Tim Horton.
+
+ * Configurations/Base.xcconfig:
+ * Configurations/DebugRelease.xcconfig:
+
+2017-06-30 Ryosuke Niwa <rniwa@webkit.org>
+
+ Ran sort-Xcode-project-file.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+
+2017-06-19 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Add a per-thread line cache
+ https://bugs.webkit.org/show_bug.cgi?id=173552
+
+ Reviewed by Darin Adler.
+
+ Previously, any thread could allocate out of any page with free lines.
+ Now, the first thread to free a line in a page owns that page's free
+ lines until the whole page becomes free.
+
+ This patch is a big speedup on multi-threaded benchmarks.
+ tree_churn --parallel gets 14% faster on a 2-core (4-hyper-core) MacBook
+ Air and 2.85X faster on 12-core (24-hyper-core) Mac Pro. Other parallel
+ benchmarks show significant but smaller speedups.
+
+ Thread affinity is a great predictor of object lifetime. The per-thread
+ line cache avoids the pathology of shuffling pages between threads,
+ turning predictable lifetimes into unpredictable lifetimes, increasing
+ fragmentation. On tree_churn --parallel, the per-thread line cache
+ increases free memory found per page scanned by 2.85X.
+
+ Free line scanning in fragmented pages is pretty expensive relative to
+ other allocate / initialize / free operations. According to Instruments,
+ on tree_churn --parallel, scanning is about 10X more expensive than
+ freeing. This explains why a 2.85X improvement in scanning efficiency
+ translates into a 2.85X overall speedup on tree_churn --parallel.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::refillAllocatorSlowCase): Pass through our line
+ cache so the Heap can fill it.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::scavenge): Scavenge our line cache.
+
+ (bmalloc::Deallocator::processObjectLog): Deleted.
+
+ * bmalloc/Deallocator.h:
+ (bmalloc::Deallocator::lineCache): Added a line cache.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::deallocateLineCache): Deallocation function for thread
+ destruction.
+
+ (bmalloc::Heap::allocateSmallPage):
+ (bmalloc::Heap::deallocateSmallLine):
+ (bmalloc::Heap::allocateSmallBumpRangesByMetadata):
+ (bmalloc::Heap::allocateSmallBumpRangesByObject): Consult the new per-thread line
+ cache for allocation and deallocation.
+
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::allocateSmallBumpRanges):
+ (bmalloc::Heap::derefSmallLine):
+
+ * bmalloc/List.h:
+ (bmalloc::List::remove): Remove has always been a logically static
+ operation. Declare it static now so that the Heap can remove a page from
+ a thread's line cache without holding a direct pointer to the cache.
+
+ * bmalloc/SmallPage.h:
+
+2017-06-10 Dan Bernstein <mitz@apple.com>
+
+ Reverted r218056 because it made the IDE reindex constantly.
+
+ * Configurations/DebugRelease.xcconfig:
+
+2017-06-10 Dan Bernstein <mitz@apple.com>
+
+ [Xcode] With Xcode 9 developer beta, everything rebuilds when switching between command-line and IDE
+ https://bugs.webkit.org/show_bug.cgi?id=173223
+
+ Reviewed by Sam Weinig.
+
+ The rebuilds were happening due to a difference in the compiler options that the IDE and
+ xcodebuild were specifying. Only the IDE was passing the -index-store-path option. To make
+ xcodebuild pass that option, too, set CLANG_INDEX_STORE_ENABLE to YES if it is unset, and
+ specify an appropriate path in CLANG_INDEX_STORE_PATH.
+
+ * Configurations/DebugRelease.xcconfig:
+
+2017-06-07 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: memory APIs don't need to be heap members
+ https://bugs.webkit.org/show_bug.cgi?id=173076
+
+ Reviewed by Sam Weinig.
+
+ Asking the OS about memory use is unrelated to the state of bmalloc's
+ heap, so it's a better separation of concerns if related code is not
+ part of the heap.
+
+ * bmalloc/AvailableMemory.cpp:
+ (bmalloc::memoryStatus):
+ * bmalloc/AvailableMemory.h:
+ (bmalloc::MemoryStatus::MemoryStatus):
+ (bmalloc::isUnderMemoryPressure):
+ (bmalloc::memoryFootprint):
+ (bmalloc::percentAvailableMemoryInUse):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::updateMemoryInUseParameters): Deleted.
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::isUnderMemoryPressure): Deleted.
+ (bmalloc::Heap::memoryFootprint): Deleted.
+ (bmalloc::Heap::percentAvailableMemoryInUse): Deleted.
+
+2017-06-06 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ struct does not accept initializer-form if member has initializers in GCC 4.9
+ https://bugs.webkit.org/show_bug.cgi?id=172974
+
+ Reviewed by Carlos Garcia Campos.
+
+ struct cannot accept initializer-form constructor (like, `ListNode<T> t { ... }`) if
+ the member of the struct has a default initializer.
+ Here is a simple snippet.
+
+ template<typename T>
+ struct Pair {
+ T* prev { nullptr };
+ T* next { nullptr };
+ };
+
+ Pair<int> pair { nullptr, nullptr }; // compile erorr in GCC 4.9.
+
+ Instead, we define a default constructor (to invoke default initializers) and a constructor
+ to accept the above initialization.
+
+ * bmalloc/List.h:
+ (bmalloc::ListNode::ListNode):
+ (bmalloc::List::iterator::iterator):
+
+2017-06-06 Geoffrey Garen <ggaren@apple.com>
+
+ Try to fix the GTK build.
+
+ Unreviewed.
+
+ * bmalloc/List.h:
+ (bmalloc::List::List):
+
+2017-06-05 Geoffrey Garen <ggaren@apple.com>
+
+ Try to fix the GTK build.
+
+ Unreviewed.
+
+ * bmalloc/List.h:
+
+2017-06-02 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Small and large objects should share memory
+ https://bugs.webkit.org/show_bug.cgi?id=172880
+ <rdar://problem/31494732>
+
+ Reviewed by Sam Weinig.
+
+ This reduces our high water mark memory usage on JetStream on macOS
+ by 10%-20%. It also has the nice side effect that we can free small
+ object metadata after returning from a high water mark.
+
+ No change in throughput.
+
+ Our old algorithm allocated small object chunks and large objects in
+ segregated virtual memory and never recycled addresses between them.
+ This provided a slight security benefit because we could apply guard
+ pages between the segregated ranges and we would never reuse the same
+ virtual address for object and metadata memory.
+
+ Our new algorithm allocates small object chunks from the large object
+ allocator. This naturally recycles memory between small chunks and large
+ objects, and between small chunks of different page classes. This allows
+ us to shift memory between allocation types as a program moves between
+ different phases of allocation, and to delete small object chunk metadata
+ when a program shrinks back from a high water mark.
+
+ Two intuitions I had about memory use turned out to be backwards in
+ this context:
+
+ (1) I thought that this optimization would work because it allowed you to
+ allocate and free a 4MB object and then reuse that large allocation to
+ service small allocations. In practice, the common benefit seems to be
+ the opposite: After you allocate and free many small objects, you can
+ stitch them together to allocate a large object without growing the heap.
+
+ (2) I thought that it would be more memory-efficient to allocate
+ fine-grained pages from the large object allocator. In practice, giving
+ the large object allocator too many arbitrarily-sized ranges to manage
+ leads to fragmentation. Meanwhile, segregated fit is a powerful memory
+ optimization. So, it's best to return small object memory to the large
+ allocator only when a whole small object chunk is free.
+
+ * bmalloc/Chunk.h:
+ (bmalloc::Chunk::ref):
+ (bmalloc::Chunk::deref):
+ (bmalloc::Chunk::refCount):
+ (bmalloc::Chunk::freePages): We keep a free list per chunk and refcount
+ each chunk so we can notice when a chunk becomes empty, and return it
+ to the large allocator.
+
+ (bmalloc::forEachPage): A new helper function for iterating the pages
+ in a Chunk.
+
+ (bmalloc::Chunk::Chunk): Use forEachPage instead of manual iteration.
+ Use { } initialization because we don't get zero-initialized by the OS
+ anymore.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::concurrentScavenge):
+ (bmalloc::Heap::scavenge): Don't bother unlocking while scavenging. I
+ wasn't able to show it to be a consistent speedup. A more promising
+ approach, if we find a motivating example, is for the scavenger to give
+ up and return early if any other client is waiting on the lock.
+
+ (bmalloc::Heap::allocateSmallChunk): New helper function for allocating
+ a small chunk. It allocates through the large allocator to facilitate
+ sharing. We still allocate a chunk at a time instead of a page at a time.
+ Surprisingly, more precise page-at-a-time allocation is worse for memory
+ use because of fragmentation. Segregated fit is a powerful optimization.
+
+ (bmalloc::Heap::deallocateSmallChunk): New helper function for deallocating
+ a small chunk.
+
+ (bmalloc::Heap::allocateSmallPage): Updated for new APIs.
+
+ (bmalloc::Heap::deallocateSmallLine): Updated for new APIs. Note that
+ we cache one free chunk per page class. This avoids churn in the large
+ allocator when you free(malloc(X)).
+
+ (bmalloc::Heap::allocateSmallBumpRangesByMetadata):
+ (bmalloc::Heap::allocateSmallBumpRangesByObject):
+ (bmalloc::Heap::tryAllocateLarge):
+ (bmalloc::Heap::scavengeSmallPages): Deleted.
+ (bmalloc::Heap::scavengeLargeObjects): Deleted.
+ * bmalloc/Heap.h:
+
+ * bmalloc/LargeMap.h:
+ (bmalloc::LargeMap::begin):
+ (bmalloc::LargeMap::end): Added iteration helpers for scavenging.
+
+ * bmalloc/LargeRange.h:
+ (bmalloc::LargeRange::physicalSize): Added a comment about something
+ that I confused myself about in this patch.
+
+ * bmalloc/List.h:
+ (bmalloc::List::iterator::operator*):
+ (bmalloc::List::iterator::operator->):
+ (bmalloc::List::iterator::operator!=):
+ (bmalloc::List::iterator::operator++):
+ (bmalloc::List::begin):
+ (bmalloc::List::end):
+ (bmalloc::List::pushFront):
+ (bmalloc::List::remove):
+ (bmalloc::ListNode::ListNode): Deleted. Added iteration helpers for
+ scavenging. Changed the default state of a Node to null pointers instead
+ of self pointers to distinguish the null node from the empty node for
+ easier debugging.
+
+ * bmalloc/Sizes.h: Changed the chunk size to 1MB to increase the chances
+ of a chunk becoming free and recyclable.
+
+ * bmalloc/SmallPage.h:
+ (bmalloc::SmallPage::hasPhysicalPages):
+ (bmalloc::SmallPage::setHasPhysicalPages): Track physical state by page
+ instead of implicitly by which list a page is in. It's simpler not
+ to have to move chunks and pages between physical vs virtual lists.
+
+ (bmalloc::SmallPage::SmallPage): Deleted.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::tryAllocateLargeChunk):
+ (bmalloc::VMHeap::allocateSmallChunk): Deleted.
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateSmallPage): Deleted.
+ (bmalloc::VMHeap::deallocateSmallPage): Deleted. Small chunk allocation
+ just forwards to the large allocator now.
+
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::scavenge):
+
+2017-05-28 Dan Bernstein <mitz@apple.com>
+
+ [Xcode] ALWAYS_SEARCH_USER_PATHS is set to YES
+ https://bugs.webkit.org/show_bug.cgi?id=172691
+
+ Reviewed by Tim Horton.
+
+ * Configurations/Base.xcconfig: Set ALWAYS_SEARCH_USER_PATHS to NO.
+
+2017-05-25 Geoffrey Garen <ggaren@apple.com> and Michael Saboff <msaboff@apple.com>
+
+ bmalloc: scavenger runs too much on JetStream
+ https://bugs.webkit.org/show_bug.cgi?id=172373
+
+ Reviewed by Geoffrey Garen.
+
+ Instruments says that JetStream on macOS spends about 3% of its time in
+ madvise.
+
+ In <https://bugs.webkit.org/show_bug.cgi?id=160098>, Ben saw some
+ evidence that madvise was the reason that switching to bmalloc for
+ DFG::Node allocations was a slowdown the first time around.
+
+ In <https://bugs.webkit.org/show_bug.cgi?id=172124>, Michael saw that
+ scavening policy can affect JetStream.
+
+ Intuitively, it seems wrong for the heap to idle shrink during hardcore
+ benchmarking.
+
+ The strategy here is to back off in response to any heap growth event,
+ and to wait 2s instead of 0.5s for heap growth to take place -- but we
+ scavenge immediately in response to critical memory pressure, to avoid
+ jetsam.
+
+ One hole in this strategy is that a workload with a perfectly
+ unfragmented heap that allocates and deallocates ~16kB every 2s will
+ never shrink its heap. This doesn't seem to be a problem in practice.
+
+ This looks like a 2% - 4% speedup on JetStream on Mac Pro and MacBook Air.
+
+ * bmalloc/AsyncTask.h:
+ (bmalloc::AsyncTask::willRun):
+ (bmalloc::AsyncTask::willRunSoon):
+ (bmalloc::Function>::AsyncTask):
+ (bmalloc::Function>::run):
+ (bmalloc::Function>::runSoon):
+ (bmalloc::Function>::threadRunLoop):
+ (bmalloc::Function>::runSlowCase): Deleted. Added a "run soon" state
+ so that execution delay is modeled directly instead of implicitly
+ through sleep events. This enables the Heap to issue a "run now" event
+ at any moment in response ot memory pressure.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap): Don't call into our own API -- that's a layering
+ violation.
+
+ (bmalloc::Heap::updateMemoryInUseParameters): No need for
+ m_scavengeSleepDuration anymore.
+
+ (bmalloc::Heap::concurrentScavenge): Added a back-off policy when the
+ heap is growing.
+ (bmalloc::Heap::scavenge):
+
+ (bmalloc::Heap::scavengeSmallPages):
+ (bmalloc::Heap::scavengeLargeObjects): Don't try to give up in the middle
+ of a scavenge event. Our new backoff policy supplants that design. Also,
+ it's easier to profile and understand scavenging behavior if it always
+ runs to completion once started.
+
+ (bmalloc::Heap::scheduleScavenger):
+ (bmalloc::Heap::scheduleScavengerIfUnderMemoryPressure): Added a
+ synchronous amortized check for memory pressure. This check has the
+ benefit that it runs immediately during high rates of heap activity,
+ so we can detect memory pressure right away and wake the scavenger
+ instead of waiting for the scavenger to wake up.
+
+ (bmalloc::Heap::allocateSmallPage):
+ (bmalloc::Heap::deallocateSmallLine):
+ (bmalloc::Heap::splitAndAllocate):
+ (bmalloc::Heap::tryAllocateLarge):
+ (bmalloc::Heap::shrinkLarge):
+ (bmalloc::Heap::deallocateLarge):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::isUnderMemoryPressure):
+ * bmalloc/Sizes.h:
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::deallocateSmallPage):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::scavenge): Updated for API changes above.
+
+2017-05-17 Michael Saboff <msaboff@apple.com>
+
+ [iOS] The Garbage Collector shouldn't rely on the bmalloc scavenger for up to date memory footprint info (172186)
+ https://bugs.webkit.org/show_bug.cgi?id=172186
+
+ Reviewed by Geoffrey Garen.
+
+ The calls memoryFootprint() and percentAvailableMemoryInUse() now make a system call to get
+ the current memory footprint value.
+
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::memoryFootprint):
+ (bmalloc::Heap::percentAvailableMemoryInUse):
+
+2017-05-16 Michael Saboff <msaboff@apple.com>
+
+ REGRESSION(r216763): JetStream is 1% slower on Mac
+ https://bugs.webkit.org/show_bug.cgi?id=172124
+
+ Reviewed by Filip Pizlo.
+
+ It appears that changing maxScavengeSleepDuration from 512 to 250ms in r216763 is
+ responsible for the regression.
+
+ * bmalloc/Sizes.h:
+
+2017-05-15 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Bump the size of the deallocator log to 512
+ https://bugs.webkit.org/show_bug.cgi?id=172143
+
+ Reviewed by Michael Saboff.
+
+ This is a speedup on parallel workloads for machines with lots of CPUs.
+
+ * bmalloc/Sizes.h:
+
+2017-05-12 Michael Saboff <msaboff@apple.com>
+
+ [iOS] Use memory footprint to dynamically adjust behavior of allocators
+ https://bugs.webkit.org/show_bug.cgi?id=171944
+
+ Reviewed by Filip Pizlo.
+
+ This change is iOS only.
+
+ After the scavenger thread completes scavenging, it asks the OS for how much total memory the
+ process is using. This information is used to update the sleep delay for the scanvenger thread,
+ as well as to provide memory in use data for other parts of the system.
+
+ The scavenger sleep time is calculated using the following quadradic equation.
+
+ scavengerSleep = 1.2*percentFreeMemory^2 - percentFreeMemory + 2
+
+ Where percentFreeMemory is between 0 and 100. The result is constrained to the values 2 and 250.
+
+ This equation empirically works out to providing a 2ms sleep time when we have less than 10%
+ memory available, 30ms when 20% is available and 250ms when 50% or more is available. In testing,
+ this exponentially agressive scavenging delay by itself reduced memory usage and made it much
+ more deterministic when used without the corresponding change in the JSC Heap.
+
+ Changed the scavenger thread to use the User Initiated QOS priority to ensure it doesn't
+ get starved.
+
+ Moved the non-Windows functionality of WTF::RAMSize() to new files AvailableMemory.{cpp,h}
+ and implemented in the function availableMemory(). That functions limits the value returned
+ on iOS to a maximum of 840MB as that is the jetsam soft memory limit.
+ Added a new API availableMemory() so that WTF::RAMSize() will use this value.
+
+ * CMakeLists.txt:
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/BPlatform.h:
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::updateMemoryInUseParameters):
+ (bmalloc::Heap::concurrentScavenge):
+ (bmalloc::Heap::scavenge):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::memoryFootprint):
+ (bmalloc::Heap::percentAvailableMemoryInUse):
+ * bmalloc/Sizes.h:
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::availableMemory):
+ (bmalloc::api::memoryFootprint):
+ (bmalloc::api::percentAvailableMemoryInUse):
+ * bmalloc/AvailableMemory.cpp: Added.
+ (bmalloc::computeAvailableMemory):
+ (bmalloc::availableMemory):
+ * bmalloc/AvailableMemory.h: Added.
+
+2017-05-05 Joseph Pecoraro <pecoraro@apple.com>
+
+ Leaks always reports "WebKit Malloc Memory Pressure Handler" dispatch_queue/source as leaking
+ https://bugs.webkit.org/show_bug.cgi?id=171532
+
+ Reviewed by Geoffrey Garen.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ * bmalloc/Heap.h:
+ Store the dispatch_source_t in a member to avoid a false positive leak.
+
+2017-04-27 Michael Saboff <msaboff@apple.com>
+
+ bmalloc scavenger should know what page classes are allocating
+ https://bugs.webkit.org/show_bug.cgi?id=171384
+
+ Reviewed by Geoffrey Garen.
+
+ This change replaces m_isAllocatingPages with a per page class flag to track which page
+ classes are currently allocating. When scavenging, we skip page classes that are actively
+ allocating and come back to them on a subsequent pass. This reduces the amount of time it
+ takes for scavenger to free up pages as well as the total time it takes to handle all
+ page classes.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::concurrentScavenge):
+ (bmalloc::Heap::scavenge):
+ (bmalloc::Heap::scavengeSmallPages):
+ (bmalloc::Heap::scavengeLargeObjects):
+ (bmalloc::Heap::allocateSmallPage):
+ (bmalloc::Heap::splitAndAllocate):
+ (bmalloc::Heap::deallocateLarge):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::takeRequestedScavengerThreadQOSClass): Deleted.
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::deallocateSmallPage):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::scavenge):
+
+2017-04-25 Michael Saboff <msaboff@apple.com>
+
+ Call bmalloc scavenger first when handling a memory pressure event
+ https://bugs.webkit.org/show_bug.cgi?id=171289
+
+ Reviewed by Geoffrey Garen.
+
+ Registered a critical memory pressure handler. We add this handler in addition to the
+ call to release bmalloc memory in the WebCore releaseMemory handler for the case of
+ JSC API users that don't use WebCore. When both handlers are in the process, it is
+ basically a race. One will win, but the loser won't do any more work, so it is harmless.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+
+2017-04-14 Mark Lam <mark.lam@apple.com>
+
+ Update architectures in xcconfig files.
+ https://bugs.webkit.org/show_bug.cgi?id=170867
+ <rdar://problem/31628104>
+
+ Reviewed by Joseph Pecoraro.
+
+ * Configurations/Base.xcconfig:
+
+2017-04-12 Dan Bernstein <mitz@apple.com>
+
+ [Mac] Future-proof .xcconfig files
+ https://bugs.webkit.org/show_bug.cgi?id=170802
+
+ Reviewed by Tim Horton.
+
+ * Configurations/Base.xcconfig:
+ * Configurations/DebugRelease.xcconfig:
+
+2017-02-03 Ting-Wei Lan <lantw44@gmail.com>
+
+ Include cstdlib before using ::malloc and posix_memalign
+ https://bugs.webkit.org/show_bug.cgi?id=167800
+
+ Reviewed by Geoffrey Garen.
+
+ * bmalloc/DebugHeap.cpp:
+
+2017-02-01 Andreas Kling <akling@apple.com>
+
+ Implement the alwaysRunsAtBackgroundPriority WK2 setting using thread QoS.
+ <https://webkit.org/b/167387>
+ <rdar://problem/29711409>
+
+ Reviewed by Antti Koivisto.
+
+ Support changing the QoS level of the scavenger thread asynchronously through
+ a request variable. This is not the most elegant thing in the world, but since
+ threads are only allowed to change their own QoS class, our options are limited.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::concurrentScavenge):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::takeRequestedScavengerThreadQOSClass):
+ (bmalloc::Heap::setScavengerThreadQOSClass):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::setScavengerThreadQOSClass):
+
+2017-01-13 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Use a separate zone when using system malloc
+ https://bugs.webkit.org/show_bug.cgi?id=167014
+
+ Reviewed by Filip Pizlo.
+
+ Harris asked for this so he could separate Safari and WebKit memory use
+ when doing memory analysis.
+
+ This patch adds an explicit DebugHeap class that contains all our
+ code for specialized allocation with debugging.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::Allocator):
+ (bmalloc::Allocator::tryAllocate):
+ (bmalloc::Allocator::allocateImpl):
+ (bmalloc::Allocator::reallocate):
+ (bmalloc::Allocator::allocateSlowCase):
+ * bmalloc/Allocator.h: Forward to DebugHeap instead of inlining all the
+ code. This is required for our new interface, and it is also a nice
+ simplification that moves some not-very-important code out of the way.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::Deallocator):
+ (bmalloc::Deallocator::scavenge):
+ (bmalloc::Deallocator::deallocateSlowCase):
+ * bmalloc/Deallocator.h: Ditto.
+
+ * bmalloc/DebugHeap.cpp: Added.
+ (bmalloc::DebugHeap::DebugHeap):
+ (bmalloc::DebugHeap::malloc):
+ (bmalloc::DebugHeap::memalign):
+ (bmalloc::DebugHeap::realloc):
+ (bmalloc::DebugHeap::free):
+ * bmalloc/DebugHeap.h: Added. New class for overriding normal heap
+ behavior. Right now, it just adds a malloc zone and then forwards to
+ system malloc -- but we can add lots more kinds of debug heaps in the
+ future if we find them useful.
+
+ * bmalloc/Environment.cpp:
+ (bmalloc::Environment::Environment):
+ (bmalloc::Environment::computeIsDebugHeapEnabled):
+ (bmalloc::Environment::computeIsBmallocEnabled): Deleted.
+ * bmalloc/Environment.h:
+ (bmalloc::Environment::isDebugHeapEnabled):
+ (bmalloc::Environment::isBmallocEnabled): Deleted. Renamed everything to
+ reflect our new use of DebugHeap.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::debugHeap): Updated to use DebugHeap.
+ (bmalloc::Heap::environment): Deleted.
+
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::isEnabled): Updated to use DebugHeap.
+
+2016-12-15 Myles C. Maxfield <mmaxfield@apple.com>
+
+ Sort Xcode project files
+ https://bugs.webkit.org/show_bug.cgi?id=165937
+
+ Reviewed by Simon Fraser.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+
+2016-12-08 David Kilzer <ddkilzer@apple.com>
+
+ Always check the return value of pthread_key_create()
+ <https://webkit.org/b/165274>
+
+ Reviewed by Darin Adler.
+
+ * bmalloc/PerThread.h:
+ (bmalloc::PerThreadStorage::init): Call BCRASH() if
+ pthread_key_create() returns an error. The error code will be
+ stored in a register available in a crash log, so no need to log
+ the value explicitly.
+
+2016-12-06 Alexey Proskuryakov <ap@apple.com>
+
+ Correct SDKROOT values in xcconfig files
+ https://bugs.webkit.org/show_bug.cgi?id=165487
+ rdar://problem/29539209
+
+ Reviewed by Dan Bernstein.
+
+ Fix suggested by Dan Bernstein.
+
+ * Configurations/DebugRelease.xcconfig:
+
+2016-11-29 Andy Estes <aestes@apple.com>
+
+ [Cocoa] Enable two clang warnings recommended by Xcode
+ https://bugs.webkit.org/show_bug.cgi?id=164498
+
+ Reviewed by Mark Lam.
+
+ * Configurations/Base.xcconfig: Enabled CLANG_WARN_INFINITE_RECURSION and CLANG_WARN_SUSPICIOUS_MOVE.
+
+2016-11-10 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc threads should have a non-default QoS
+ https://bugs.webkit.org/show_bug.cgi?id=164612
+
+ Reviewed by Filip Pizlo.
+
+ * bmalloc/AsyncTask.h:
+ (bmalloc::Function>::threadEntryPoint): Request user-interactive quality
+ of service because user-interactive tasks use malloc.
+
+2016-10-20 Mark Lam <mark.lam@apple.com>
+
+ bmalloc api should crash on failure to allocate when !isBmallocEnabled.
+ https://bugs.webkit.org/show_bug.cgi?id=163766
+
+ Reviewed by Keith Miller and Filip Pizlo.
+
+ We want to crash in bmalloc on failure to allocate even when !isBmallocEnabled.
+ This is so that failures to allocate memory will manifest as crashes with a
+ unique signature (i.e. as a SIGTRAP on release builds, or as a write to illegal
+ address 0xbbadbeef on debug builds) and the crash will manifest inside bmalloc.
+ This distinguishes allocation failures from other crashing bugs that manifest as
+ SIGSEGVs due to random pointer dereferences in the clients of bmalloc.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocateImpl):
+ (bmalloc::Allocator::reallocate):
+ (bmalloc::Allocator::allocateSlowCase):
+
+2016-09-26 Yoshiaki Jitsukawa <Yoshiaki.Jitsukawa@sony.com>
+
+ Avoid implicit conversion from iterator to pointer
+ https://bugs.webkit.org/show_bug.cgi?id=162482
+
+ Reviewed by Geoffrey Garen.
+
+ Not every STL supporting such conversion, we should get a pointer explicitly.
+
+ * bmalloc/Chunk.h:
+ (bmalloc::Chunk::lines):
+ (bmalloc::Chunk::pages):
+ * bmalloc/FixedVector.h:
+ (bmalloc::FixedVector::begin):
+
+2016-08-31 Filip Pizlo <fpizlo@apple.com>
+
+ Butterflies should be allocated in Auxiliary MarkedSpace instead of CopiedSpace and we should rewrite as much of the GC as needed to make this not a regression
+ https://bugs.webkit.org/show_bug.cgi?id=160125
+
+ Reviewed by Geoffrey Garen and Keith Miller.
+
+ I needed to tryMemalign, so I added such a thing.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocate):
+ (bmalloc::Allocator::tryAllocate):
+ (bmalloc::Allocator::allocateImpl):
+ * bmalloc/Allocator.h:
+ * bmalloc/Cache.h:
+ (bmalloc::Cache::tryAllocate):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::tryMemalign):
+
+2016-08-30 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ Unreviewed, build fix for GCC ports
+
+ std::forward is declared in <utility> header.
+
+ * bmalloc/ScopeExit.h:
+
+2016-08-30 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: speed up the lock slow path
+ https://bugs.webkit.org/show_bug.cgi?id=161058
+
+ Unreviewed roll-in - with regression fixed.
+
+ Revert to using yield() instead of swtch() because very low priority
+ background tasks can cause priority inversion and deadlock. In the
+ network process, that happened with com.apple.WebKit.Cache.Storage.serialBackground.
+
+ Still a big speedup on MallocBench.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/ScopeExit.h: Added.
+ (bmalloc::ScopeExit::ScopeExit):
+ (bmalloc::ScopeExit::~ScopeExit):
+ (bmalloc::makeScopeExit):
+ * bmalloc/StaticMutex.cpp:
+ (bmalloc::StaticMutex::lockSlowCase):
+ * bmalloc/StaticMutex.h:
+ (bmalloc::StaticMutex::init):
+
+2016-08-26 Geoffrey Garen <ggaren@apple.com>
+
+ Unreviewed build fix.
+
+ Fix the CMake build.
+
+ * CMakeLists.txt:
+
+2016-08-26 Geoffrey Garen <ggaren@apple.com>
+
+ Renamed XLarge* => Large*
+ https://bugs.webkit.org/show_bug.cgi?id=161261
+
+ Reviewed by Andreas Kling.
+
+ XLarge is not a thing anymore: We just have Small and Large.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::splitAndAllocate):
+ (bmalloc::Heap::tryAllocateLarge):
+ (bmalloc::Heap::shrinkLarge):
+ (bmalloc::Heap::deallocateLarge):
+ * bmalloc/Heap.h:
+ * bmalloc/LargeMap.cpp: Copied from Source/bmalloc/bmalloc/XLargeMap.cpp.
+ (bmalloc::LargeMap::remove):
+ (bmalloc::LargeMap::add):
+ (bmalloc::XLargeMap::remove): Deleted.
+ (bmalloc::XLargeMap::add): Deleted.
+ * bmalloc/LargeMap.h: Copied from Source/bmalloc/bmalloc/XLargeMap.h.
+ (bmalloc::LargeMap::ranges):
+ (bmalloc::XLargeMap::ranges): Deleted.
+ * bmalloc/LargeRange.h: Copied from Source/bmalloc/bmalloc/XLargeRange.h.
+ (bmalloc::LargeRange::LargeRange):
+ (bmalloc::LargeRange::operator<):
+ (bmalloc::canMerge):
+ (bmalloc::merge):
+ (bmalloc::LargeRange::split):
+ (bmalloc::XLargeRange::XLargeRange): Deleted.
+ (bmalloc::XLargeRange::operator<): Deleted.
+ (bmalloc::XLargeRange::split): Deleted.
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::tryAllocateLargeChunk):
+ * bmalloc/VMHeap.h:
+ * bmalloc/XLargeMap.cpp: Removed.
+ * bmalloc/XLargeMap.h: Removed.
+ * bmalloc/XLargeRange.h: Removed.
+
+2016-08-26 Gavin Barraclough <barraclough@apple.com>
+
+ bmalloc: speed up the lock slow path
+ https://bugs.webkit.org/show_bug.cgi?id=161058
+
+ Unreviewed rollout - this caused regressions <rdar://problem/28026089>.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/ScopeExit.h: Removed.
+ * bmalloc/StaticMutex.cpp:
+ (bmalloc::StaticMutex::lockSlowCase):
+ * bmalloc/StaticMutex.h:
+ (bmalloc::StaticMutex::init):
+ * bmalloc/ThreadSwitch.h: Removed.
+
+2016-08-24 Andreas Kling <akling@apple.com>
+
+ Add bmalloc::api::isEnabled().
+ <https://webkit.org/b/160534>
+
+ Reviewed by Joseph Pecoraro.
+
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::isEnabled):
+
+2016-08-24 Filip Pizlo <fpizlo@apple.com>
+
+ Unreviewed, roll out r204901, r204897, r204866, r204856, r204854.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocate):
+ (bmalloc::Allocator::tryAllocate): Deleted.
+ (bmalloc::Allocator::allocateImpl): Deleted.
+ * bmalloc/Allocator.h:
+ * bmalloc/Cache.h:
+ (bmalloc::Cache::tryAllocate): Deleted.
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::tryMemalign): Deleted.
+
+2016-08-12 Filip Pizlo <fpizlo@apple.com>
+
+ Butterflies should be allocated in Auxiliary MarkedSpace instead of CopiedSpace and we should rewrite as much of the GC as needed to make this not a regression
+ https://bugs.webkit.org/show_bug.cgi?id=160125
+
+ Reviewed by Geoffrey Garen.
+
+ I needed to tryMemalign, so I added such a thing.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocate):
+ (bmalloc::Allocator::tryAllocate):
+ (bmalloc::Allocator::allocateImpl):
+ (bmalloc::Allocator::reallocate):
+ * bmalloc/Allocator.h:
+ * bmalloc/Cache.h:
+ (bmalloc::Cache::allocate):
+ (bmalloc::Cache::tryAllocate):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::malloc):
+ (bmalloc::api::tryMemalign):
+ (bmalloc::api::memalign):
+
+2016-08-22 Yusuke Suzuki <utatane.tea@gmail.com>
+
+ Unreviewed, build fix on GCC environment
+ https://bugs.webkit.org/show_bug.cgi?id=161058
+
+ std::forward is declared in <utility> header.
+
+ * bmalloc/ScopeExit.h:
+
+2016-08-22 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: speed up the lock slow path
+ https://bugs.webkit.org/show_bug.cgi?id=161058
+
+ Reviewed by Filip Pizlo.
+
+ It is generally accepted practice that a lock should yield instead of
+ spinning when a lock acquisition fails, to avoid wasting CPU and power.
+
+ There are two problems with this generally accepted practice:
+
+ (1) It's a fallacy that yielding is free. In reality, yielding itself
+ consumes CPU and power -- by performing a syscall, running the OS
+ scheduler, and possibly performing a context switch. (Instruments
+ traces of MallocBench show the cost of yielding.) Therefore, spinning a
+ little to avoid yielding can actually *save* CPU and power.
+
+ (2) std::this_thread_yield() on Darwin is way too aggressive: It not only
+ yields but also depresses your priority to absolute zero for 10ms. A
+ recent PLT trace showed a few spots where the main thread just gave up
+ on loading and rendering a page for 10ms so an unimportant background
+ task could run.
+
+ To correct these problems, this patch adds a little bit of spinning to
+ the bmalloc lock slow path.
+
+ Below are performance results on various CPUs.
+
+ Mac Pro (12 hyperthreaded cores = 24 threads):
+
+ Baseline Patch Δ
+ Execution Time:
+ message_one 173ms 173ms
+ message_many 953ms 927ms ^ 1.03x faster
+ churn --parallel 60ms 41ms ^ 1.46x faster
+ list_allocate --parallel 224ms 143ms ^ 1.57x faster
+ tree_allocate --parallel 1,190ms 758ms ^ 1.57x faster
+ tree_churn --parallel 1,517ms 906ms ^ 1.67x faster
+ facebook --parallel 6,519ms 4,580ms ^ 1.42x faster
+ reddit --parallel 5,097ms 3,411ms ^ 1.49x faster
+ flickr --parallel 4,903ms 3,501ms ^ 1.4x faster
+ theverge --parallel 6,641ms 4,505ms ^ 1.47x faster
+
+ <geometric mean> 1,158ms 832ms ^ 1.39x faster
+ <arithmetic mean> 2,728ms 1,895ms ^ 1.44x faster
+ <harmonic mean> 332ms 240ms ^ 1.38x faster
+
+ MacBook Air (2 hyperthreaded cores = 4 threads):
+
+ Baseline Patch Δ
+ Execution Time:
+ message_one 911ms 907ms ^ 1.0x faster
+ message_many 515ms 513ms ^ 1.0x faster
+ churn --parallel 132ms 134ms ! 1.02x slower
+ list_allocate --parallel 104ms 102ms ^ 1.02x faster
+ tree_allocate --parallel 117ms 111ms ^ 1.05x faster
+ tree_churn --parallel 154ms 151ms ^ 1.02x faster
+ facebook --parallel 719ms 687ms ^ 1.05x faster
+ reddit --parallel 382ms 341ms ^ 1.12x faster
+ flickr --parallel 372ms 345ms ^ 1.08x faster
+ theverge --parallel 489ms 444ms ^ 1.1x faster
+
+ <geometric mean> 299ms 287ms ^ 1.04x faster
+ <arithmetic mean> 390ms 374ms ^ 1.04x faster
+ <harmonic mean> 227ms 220ms ^ 1.03x faster
+
+ iPad (2 cores = 2 threads):
+
+ [ Doesn't run Ruby, so no pretty subtest output. ]
+
+ Baseline Patch Δ
+ Execution Time: 174.14ms 171.5ms ^ 1.02x faster
+
+ * bmalloc.xcodeproj/project.pbxproj:
+
+ * bmalloc/ScopeExit.h: Added. A barebones very wimpy version of
+ WTF::ScopeExit.
+ (bmalloc::ScopeExit::ScopeExit):
+ (bmalloc::ScopeExit::~ScopeExit):
+ (bmalloc::makeScopeExit):
+
+ * bmalloc/StaticMutex.cpp:
+ (bmalloc::StaticMutex::lockSlowCase): Spin before yielding -- that's the
+ speedup. Don't spin if another CPU is already spinning. In theory, more
+ than one spinner accomplishes nothing, and I found that there's a cutoff
+ around 8 or 16 spinners that becomes performance negative on Mac Pro.
+
+ (Note: Another way to accomplish a similar result, if you don't want to
+ use a bit of state in the lock, is to spin for a random duration between
+ 0 and aLot. I tested a version of WTF::WeakRandom with unsynchronized
+ static state and it worked great. But I ultimately opted for the explicit
+ bit because I thought it was clearer.)
+
+ * bmalloc/StaticMutex.h:
+ (bmalloc::StaticMutex::init): Initialize our new bit.
+
+ * bmalloc/ThreadSwitch.h: Added.
+ (bmalloc::threadSwitch): Don't call yield() on Darwin because it's too
+ aggressive. swtch() does what we want: Go run something else, without
+ any other side-effects.
+
+2016-08-03 Geoffrey Garen <ggaren@apple.com>
+
+ [bmalloc] Merging of XLargeRanges can leak the upper range
+ https://bugs.webkit.org/show_bug.cgi?id=160403
+
+ Reviewed by Michael Saboff.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::scavengeLargeObjects): Don't use removePhysical().
+ Recorded physical size is a performance optimization. It is not the
+ truth. So it might be zero even if a range contains physical pages.
+
+ Instead, iterate each range in the map unconditionally.
+
+ The map can shrink when we release the lock, so we must clamp our
+ iterator each time through the loop.
+
+ The map can grow when we release the lock, but we don't care because
+ growth restarts the scavenger from the beginning.
+
+ * bmalloc/XLargeMap.cpp:
+ (bmalloc::XLargeMap::removePhysical): Deleted. Not used anymore.
+
+ * bmalloc/XLargeMap.h:
+ (bmalloc::XLargeMap::ranges): Added direct access for the sake of
+ scavengeLargeObjects. (This violates our naming conventions -- I'll do
+ a rename in a follow-up patch.)
+
+2016-07-13 Enrica Casucci <enrica@apple.com>
+
+ Update supported platforms in xcconfig files to match the sdk names.
+ https://bugs.webkit.org/show_bug.cgi?id=159728
+
+ Reviewed by Tim Horton.
+
+ * Configurations/Base.xcconfig:
+
+2016-07-11 Geoffrey Garen <ggaren@apple.com>
+
+ Crash due to abort() calling libc++.1.dylib: std::__1::thread::detach()
+ https://bugs.webkit.org/show_bug.cgi?id=159655
+
+ Reviewed by Sam Weinig.
+
+ It's not entirely clear what was happening in these crashes, but our
+ use of detach() was 100% forward-looking, so we can just remove it for
+ now.
+
+ This patch removes the ability for the scavenger owner to die before
+ the scavenger thread dies (which was unused) and also removes the
+ ability for the scavenger thread to exit (which was used, but we
+ messed up and did thread joining lazily, so we never got any benefit
+ from thread exit.)
+
+ We can add these features back when we need them, and make them work then.
+
+ * bmalloc/AsyncTask.h:
+ (bmalloc::Function>::AsyncTask): We start out in the running state now
+ because we know that starting our thread will run it.
+
+ (bmalloc::Function>::~AsyncTask): We don't support destruction anymore.
+
+ (bmalloc::Function>::runSlowCase): I removed the Exited state.
+
+ (bmalloc::Function>::threadRunLoop): I removed the Exited and
+ ExitRequested states.
+
+ * bmalloc/Heap.h:
+
+ * bmalloc/VMHeap.h:
+
+2016-06-12 David Kilzer <ddkilzer@apple.com>
+
+ Crash in com.apple.WebKit.WebContent at std::__1::__call_once_proxy<std::__1::tuple<CrashReporterSupportLibrary()::$_0&&> >
+ <https://webkit.org/b/158660>
+ <rdar://problem/25652686>
+
+ Reviewed by Darin Adler.
+
+ * bmalloc/Logging.cpp: Switch to use
+ BSOFT_LINK_PRIVATE_FRAMEWORK() to link
+ CrashReporterSupport.framework.
+ * bmalloc/darwin/BSoftLinking.h:
+ (BSOFT_LINK_PRIVATE_FRAMEWORK): Rename from BSOFT_LINK_FRAMEWORK.
+ Switch to use /System/Library/PrivateFrameworks/.
+
+2016-06-11 David Kilzer <ddkilzer@apple.com>
+
+ Implement logging for RELEASE_BASSERT_WITH_MESSAGE() in BAssert.h
+ <http://webkit.org/b/155992>
+
+ Reviewed by Geoff Garen.
+
+ * bmalloc/BAssert.h:
+ (BLOG_ERROR): Add method to always log error messages.
+ (RELEASE_BASSERT_WITH_MESSAGE): Use BLOG_ERROR() to implement
+ logging in Debug builds.
+ * bmalloc/BPlatform.h:
+ (BPLATFORM_MAC): Add.
+ (BUSE): Add BUSE() macro.
+ (BATTRIBUTE_PRINTF): Add.
+ (BUSE_OS_LOG): Add.
+ * bmalloc/Logging.cpp:
+ (bmalloc::reportAssertionFailureWithMessage): Add. Logs to
+ stderr.
+ * bmalloc/Logging.h:
+ (bmalloc::reportAssertionFailureWithMessage): Add declaration.
+
+2016-06-07 Pranjal Jumde <pjumde@apple.com>
+
+ Prevents integer overflow in Vector.h
+ https://bugs.webkit.org/show_bug.cgi?id=158455
+ <rdar://problem/20235469>
+
+ Reviewed by Mark Lam.
+
+ * bmalloc/Vector.h:
+ (bmalloc::Vector<T>::reallocateBuffer):
+
+2016-05-27 Konstantin Tokarev <annulen@yandex.ru>
+
+ [cmake] Deduplicated bmalloc/Zone.cpp handling.
+ https://bugs.webkit.org/show_bug.cgi?id=158154
+
+ Reviewed by Alex Christensen.
+
+ File bmalloc/Zone.cpp is required on Darwin irrespectively from what
+ port is being built.
+
+ Also I removed WEBKIT_INCLUDE_CONFIG_FILES_IF_EXISTS() because it's
+ unlikely that bmalloc will ever need port-specific customizations (as
+ opposed to OS-specific customizations which should be done in
+ CMakeLists.txt).
+
+ * CMakeLists.txt: Added bmalloc/Zone.cpp for Darwin.
+ * PlatformGTK.cmake: Removed.
+ * PlatformMac.cmake: Removed.
+
+2016-05-22 Brady Eidson <beidson@apple.com>
+
+ Move to C++14.
+ https://bugs.webkit.org/show_bug.cgi?id=157948
+
+ Reviewed by Michael Catanzaro.
+
+ * Configurations/Base.xcconfig:
+
+2016-05-17 Geoffrey Garen <ggaren@apple.com>
+
+ REGRESSION: JetStream crashes on some iPhones
+ https://bugs.webkit.org/show_bug.cgi?id=157814
+
+ Reviewed by Michael Saboff.
+
+ * bmalloc/Sizes.h: Reduce smallMax to 32kB.
+
+ Previous justification for 64kB was:
+
+ * bmalloc/Sizes.h: Upped smallMax to 64kB. Upping to 32kB is pretty
+ reasonable, since sizes between 16kB and 32kB share page sizes. I went
+ all the way up to 64kB because the GC uses 64kB blocks, and also just
+ for extra padding to ensure that large allocations are indeed rare.
+
+ It turns out that the bump to 64kB substantially increases our memory
+ high water mark on JetStream, leading to jetsam crashes. Also, there
+ doesn't seem to be a practical performance problem to putting objects in
+ the (32kB - 64kB) range in the large allocator.
+
+2016-05-16 Geoffrey Garen <ggaren@apple.com>
+
+ REGRESSION (200035): changes in "WebKit Malloc" VM regions are causing 'leaks' to spew "Failed to map remote region" messages
+ https://bugs.webkit.org/show_bug.cgi?id=157764
+
+ Reviewed by Gavin Barraclough.
+
+ We need to allow for guard pages and only report unguarded pages to the
+ leaks tool -- otherwise, it will try to remote map our guarded pages,
+ and crash.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::tryAllocateLargeChunk):
+ (bmalloc::VMHeap::allocateSmallChunk): Adopt the new API for reporting
+ a range instead of a Chunk*, and report the unguarded range.
+
+ This also fixes a separate bug -- very large allocations would not
+ fully participate in pointer scanning because they would only report 2MB
+ (chunkSize) in size. This could cause false-positive leak reports.
+
+ * bmalloc/Zone.cpp:
+ (bmalloc::enumerator): Updated to scan ranges instead of fixed-sized
+ Chunk pointers.
+
+ * bmalloc/Zone.h:
+ (bmalloc::Zone::ranges):
+ (bmalloc::Zone::addRange): Store ranges instead of fixed-sized Chunk
+ pointers because our VM ranges have variable sizes -- both due to guard
+ pages and due to large allocations.
+
+ (bmalloc::Zone::chunks): Deleted.
+ (bmalloc::Zone::addChunk): Deleted.
+
+2016-05-10 David Kilzer <ddkilzer@apple.com>
+
+ bmalloc should automatically disable itself when ThreadSanitizer is used
+ <https://webkit.org/b/157527>
+
+ Reviewed by Michael Catanzaro.
+
+ * bmalloc/Environment.cpp:
+ (bmalloc::isASanEnabled): Rename to isSanitizerEnabled.
+ (bmalloc::isSanitizerEnabled): Rename from isASanEnabled. Add
+ support for detecting ThreadSanitizer.
+ (bmalloc::Environment::computeIsBmallocEnabled): Switch from
+ isASanEnabled to isSanitizerEnabled.
+
+2016-05-03 Geoffrey Garen <ggaren@apple.com>
+
+ Assertion failure in bmalloc::vmRevokePermissions(void*, unsigned long).
+ https://bugs.webkit.org/show_bug.cgi?id=157047
+
+ Reviewed by Filip Pizlo.
+
+ Renamed roundUpToMultipleOfSloppy => roundUpToMultipleOfNonPowerOfTwo.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::roundUpToMultipleOfNonPowerOfTwo):
+ (bmalloc::roundUpToMultipleOfSloppy): Deleted.
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::allocateSmallChunk):
+
+2016-05-03 Geoffrey Garen <ggaren@apple.com>
+
+ Assertion failure in bmalloc::vmRevokePermissions(void*, unsigned long).
+ https://bugs.webkit.org/show_bug.cgi?id=157047
+
+ Reviewed by Filip Pizlo.
+
+ The previous fix aligned the guard page sizes correctly but forgot to
+ align the guard page start address correctly.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::roundUpToMultipleOfSloppy): Use a new helper method to round
+ up when not working with a power of two, instead of writing out the
+ math by hand.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::allocateSmallChunk): Make sure to round up the guard
+ page start address in addition to its size. Assert at the very end to
+ try to catch more bugs.
+
+2016-04-27 Geoffrey Garen <ggaren@apple.com>
+
+ Assertion failure in bmalloc::vmRevokePermissions(void*, unsigned long).
+ https://bugs.webkit.org/show_bug.cgi?id=157047
+
+ Reviewed by Darin Adler.
+
+ * bmalloc/Chunk.h:
+ (bmalloc::Chunk::Chunk):
+ (bmalloc::Chunk::get):
+ (bmalloc::Chunk::offset):
+ (bmalloc::Chunk::address):
+ (bmalloc::Object::Object):
+ (bmalloc::Object::address):
+ (bmalloc::Object::line):
+ (bmalloc::Chunk::object): Deleted.
+ (bmalloc::Object::begin): Deleted.
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateSmallBumpRangesByObject):
+ * bmalloc/Object.h:
+ (bmalloc::Object::chunk):
+ (bmalloc::Object::offset): Renamed begin() to address() because this is
+ not an iterator.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::allocateSmallChunk): Round up pageSize to a vmPageSize
+ multiple because pageSize might be smaller than vmPageSize, but we
+ think the VM system requires vmPageSize-aligned values.
+
+2016-04-25 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: vm allocations should plant guard pages
+ https://bugs.webkit.org/show_bug.cgi?id=156937
+
+ Rolling back in r199936 with a fix for the memory regression.
+
+2016-04-23 Gavin Barraclough <barraclough@apple.com>
+
+ bmalloc: vm allocations should plant guard pages
+ https://bugs.webkit.org/show_bug.cgi?id=156937
+
+ Rolling out - looks like this is memory regression.
+
+ * bmalloc/Object.h:
+ (bmalloc::Object::operator+):
+ (bmalloc::Object::operator<=):
+ (bmalloc::Object::operator-): Deleted.
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmDeallocate):
+ (bmalloc::vmRevokePermissions): Deleted.
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::allocateSmallChunk):
+
+2016-04-22 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: vm allocations should plant guard pages
+ https://bugs.webkit.org/show_bug.cgi?id=156937
+
+ Reviewed by Michael Saboff.
+
+ * bmalloc/Object.h:
+ (bmalloc::Object::operator-): Added a - helper.
+
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmRevokePermissions): Added a helper to revoke permissions on
+ a VM region. We use this for guard pages.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::allocateSmallChunk): Add guard pages to the start and
+ end of the chunk.
+
+ Note that we don't guard large chunks becuase we need to be able to merge
+ them. Otherwise, we will run out of virtual addresses.
+
+2016-04-22 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Constify introspect function pointer table
+ https://bugs.webkit.org/show_bug.cgi?id=156936
+
+ Reviewed by Michael Saboff.
+
+ * bmalloc/Zone.cpp:
+ (bmalloc::Zone::Zone): Declaring this function pointer table const puts
+ it in the read-only section of the binary, providing a little hardening
+ against overwriting the function pointers at runtime. (We have to
+ const_cast when assigning because the API declares a pointer to non-const,
+ but we happen to know it will never try to write through that pointer.
+ This is not my favorite API.)
+
+2016-04-19 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: fix up overflow checks
+ https://bugs.webkit.org/show_bug.cgi?id=156780
+
+ Reviewed by Mark Lam.
+
+ We used to try to avoid overflow in large object math by setting a very
+ high limit on the largest large object. But that's a bit error-prone
+ since the check is far away from the math that might overflow -- and
+ we were missing some cases.
+
+ This patch removes the limit and instead checks at each math site.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::tryAllocate):
+ (bmalloc::Allocator::allocate):
+ (bmalloc::Allocator::reallocate):
+ (bmalloc::Allocator::allocateSlowCase): Remove the limit. tryAllocateLarge
+ will check for overflow for us.
+
+ * bmalloc/Chunk.h: This ASSERT was just totally wrong.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::tryAllocateLarge): Check for overflow when adding.
+
+ * bmalloc/Sizes.h:
+
+ * bmalloc/VMAllocate.h:
+ (bmalloc::tryVMAllocate): Check for overflow when adding.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::tryAllocateLargeChunk): Check for overflow when adding.
+
+2016-04-19 Geoffrey Garen <ggaren@apple.com>
+
+ Unreviewed, try to fix an ASSERT seen on the bots.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::tryAllocateLarge): This ASSERT is supposed to be about
+ alignment, not size. Oops.
+
+2016-04-19 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Merge the large and xlarge allocators
+ https://bugs.webkit.org/show_bug.cgi?id=156734
+
+ Reviewed by Andreas Kling.
+
+ This give us better defense against worst case memory usage:
+
+ Baseline Patch Δ
+ Peak Memory:
+ nimlang 198,132kB 181,468kB ^ 1.09x smaller
+
+ It also eliminates inline metadata for large objects, fixing the
+ regression introduced in r198675, and more:
+
+ run-malloc-benchmarks Baseline:~/OpenSource/WebKitBuildBaseline/Release/ Patch:~/OpenSource/WebKitBuild/Release/
+
+ Baseline Patch Δ
+ Memory at End:
+ big 10,880kB 3,328kB ^ 3.27x smaller
+ facebook 3,112kB 2,868kB ^ 1.09x smaller
+ fragment --parallel 1,848kB 760kB ^ 2.43x smaller
+ fragment_iterate --parallel 4,908kB 776kB ^ 6.32x smaller
+ big --parallel 48,076kB 11,892kB ^ 4.04x smaller
+
+ Overall memory use looks OK:
+
+ run-malloc-benchmarks --memory_warning Baseline:~/OpenSource/WebKitBuildBaseline/Release/ Patch:~/OpenSource/WebKitBuild/Release/
+
+ Baseline Patch Δ
+ Memory at End:
+ <arithmetic mean> 13,992kB 13,987kB ^ 1.0x smaller
+
+ Overall throughput looks OK:
+
+ run-malloc-benchmarks Baseline:~/OpenSource/WebKitBuildBaseline/Release/ Patch:~/OpenSource/WebKitBuild/Release/
+
+ Baseline Patch Δ
+ Execution Time:
+ <arithmetic mean> 103ms 104ms ! 1.01x slower
+
+ We're a bit slower on the "all-out large allocations on all cores"
+ benchmark, but I think that's an OK price to pay:
+
+ Baseline Patch Δ
+ Execution Time:
+ big --parallel 125ms 136ms ! 1.09x slower
+
+ This patch net removes 1.5k lines of code. It turns out that large
+ allocations are rare, and free memory fragments are also rare, so the
+ combination is super rare, and a simple O(n) algorithm that ensures good
+ memory behavior is the best option.
+
+ Fun fact: In practice, the odds that the old code would save memory
+ were *worse* than the odds that it would contain a bug that wasted
+ memory. :)
+
+ * bmalloc.xcodeproj/project.pbxproj:
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::tryAllocate): largeMax is the new xLargeMax since
+ xLargeMax is gone now.
+
+ (bmalloc::Allocator::allocate): I moved the rounding code into allocateLarge,
+ so we don't have to do it here.
+
+ (bmalloc::Allocator::reallocate):
+ (bmalloc::Allocator::allocateSlowCase):
+ (bmalloc::Allocator::allocateXLarge): Deleted. No more XLarge case.
+
+ * bmalloc/Allocator.h:
+
+ * bmalloc/BeginTag.h: Removed.
+ * bmalloc/BoundaryTag.h: Removed.
+
+ * bmalloc/Chunk.h:
+ (bmalloc::ChunkHash::hash): Added a hash function. The best hash function
+ is a unique and monotonically increasing integer, and that's exactly what
+ we typically get from the high bits of a Chunk, since the OS allocates
+ Chunks at unique and increasing addresses.
+ (bmalloc::Chunk::boundaryTags): Deleted.
+ (bmalloc::Chunk::objectType): Deleted.
+ (bmalloc::Chunk::beginTag): Deleted.
+ (bmalloc::Chunk::endTag): Deleted.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::deallocateSlowCase): We no longer know for sure,
+ by looking at its bit pattern, whether a pointer is small or large.
+ Instead, any pointer with large alignment *might* be large, and when
+ we occasionally encounter such an object, we have to consult a hash
+ table in the Heap to find out for sure. This turns out to be just as
+ cheap in practice.
+
+ We don't deallocate large objects on the fast path anymore. We can't,
+ because large objects have out-of-line metadata now.
+
+ (bmalloc::Deallocator::deallocateXLarge): Deleted.
+
+ * bmalloc/Deallocator.h:
+ (bmalloc::Deallocator::deallocateFastCase): See deallocateSlowCase.
+
+ * bmalloc/EndTag.h: Removed.
+ * bmalloc/FreeList.cpp: Removed.
+ * bmalloc/FreeList.h: Removed.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateSmallPage): Be sure to track each chunk in
+ the object type map, so we can distinguish small vs large objects.
+
+ (bmalloc::Heap::deallocateSmallLine): No need to check object type
+ because we know object type now by virtue of being on the small object
+ path.
+
+ (bmalloc::Heap::splitAndAllocate): Be sure to track each chunk in
+ the object type map, so we can distinguish small vs large objects. Large
+ objects can split across chunks, so we need to add each large object's
+ chunk as it is allocated.
+
+ (bmalloc::Heap::tryAllocateLarge):
+ (bmalloc::Heap::allocateLarge):
+ (bmalloc::Heap::isLarge):
+ (bmalloc::Heap::largeSize):
+ (bmalloc::Heap::shrinkLarge):
+ (bmalloc::Heap::deallocateLarge): Merged in existing XLarge logic for
+ large objects.
+
+ (bmalloc::Heap::scavengeXLargeObjects): Deleted.
+ (bmalloc::Heap::allocateXLarge): Deleted.
+ (bmalloc::Heap::tryAllocateXLarge): Deleted.
+ (bmalloc::Heap::xLargeSize): Deleted.
+ (bmalloc::Heap::shrinkXLarge): Deleted.
+ (bmalloc::Heap::deallocateXLarge): Deleted.
+
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::LargeObjectHash::hash):
+
+ * bmalloc/LargeObject.h: Removed.
+
+ * bmalloc/Map.h: Added.
+ (bmalloc::Map::size):
+ (bmalloc::Map::capacity):
+ (bmalloc::Map::get):
+ (bmalloc::Map::set):
+ (bmalloc::Map::remove):
+ (bmalloc::Map::shouldGrow):
+ (bmalloc::Map::shouldShrink):
+ (bmalloc::Map::find):
+ (bmalloc::Hash>::rehash): Simple hash table.
+
+ * bmalloc/Object.h:
+
+ * bmalloc/ObjectType.cpp:
+ (bmalloc::objectType):
+ * bmalloc/ObjectType.h:
+ (bmalloc::mightBeLarge): See deallocateSlowCase.
+ (bmalloc::isXLarge): Deleted.
+
+ * bmalloc/SegregatedFreeList.cpp: Removed.
+ * bmalloc/SegregatedFreeList.h: Removed.
+
+ * bmalloc/Sizes.h: Upped smallMax to 64kB. Upping to 32kB is pretty
+ reasonable, since sizes between 16kB and 32kB share page sizes. I went
+ all the way up to 64kB because the GC uses 64kB blocks, and also just
+ for extra padding to ensure that large allocations are indeed rare.
+
+ * bmalloc/SortedVector.h: Removed.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::tryAllocateLargeChunk):
+ (bmalloc::VMHeap::allocateSmallChunk):
+ (bmalloc::VMHeap::VMHeap): Deleted.
+ (bmalloc::VMHeap::allocateChunk): Deleted.
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::deallocateSmallPage):
+ (bmalloc::VMHeap::allocateLargeObject): Deleted.
+ (bmalloc::VMHeap::deallocateLargeObject): Deleted. Nixed all the boundary
+ tag logic since metadata is out of line now.
+
+ * bmalloc/VMState.h: Removed. Instead of an abstract state, we track
+ the precise amount of committed physical pages at the head of a VM
+ range. This allows us to merge aggressively without triggering an madvise
+ storm most of the time.
+
+ * bmalloc/Vector.h:
+ (bmalloc::Vector<T>::Vector):
+ (bmalloc::Vector<T>::insert):
+ (bmalloc::Vector<T>::remove):
+ (bmalloc::Vector<T>::resize): Filled out some missing helpers.
+
+ * bmalloc/XLargeMap.cpp:
+ (bmalloc::XLargeMap::remove):
+ (bmalloc::XLargeMap::add):
+ (bmalloc::XLargeMap::removePhysical):
+ (bmalloc::XLargeMap::takeFree): Deleted.
+ (bmalloc::XLargeMap::addFree): Deleted.
+ (bmalloc::XLargeMap::addAllocated): Deleted.
+ (bmalloc::XLargeMap::getAllocated): Deleted.
+ (bmalloc::XLargeMap::takeAllocated): Deleted.
+ (bmalloc::XLargeMap::shrinkToFit): Deleted.
+ (bmalloc::XLargeMap::takePhysical): Deleted.
+ (bmalloc::XLargeMap::addVirtual): Deleted.
+ * bmalloc/XLargeMap.h:
+ (bmalloc::XLargeMap::Allocation::operator<): Deleted. We don't track
+ object sizes anymore -- just free space. (The Heap tracks object sizes.)
+ We use plain old linear search for free space. (See intro.)
+
+ * bmalloc/XLargeRange.h:
+ (bmalloc::XLargeRange::physicalSize):
+ (bmalloc::XLargeRange::setPhysicalSize):
+ (bmalloc::merge):
+ (bmalloc::XLargeRange::split):
+ (bmalloc::XLargeRange::vmState): Deleted.
+ (bmalloc::XLargeRange::setVMState): Deleted. See VMState.h.
+
+2016-04-11 Fujii Hironori <Hironori.Fujii@jp.sony.com>
+
+ [CMake] Make FOLDER property INHERITED
+ https://bugs.webkit.org/show_bug.cgi?id=156460
+
+ Reviewed by Brent Fulgham.
+
+ * CMakeLists.txt:
+ Set FOLDER property as a directory property not a target property
+
+2016-04-08 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: stress_aligned test fails if you increase smallMax
+ https://bugs.webkit.org/show_bug.cgi?id=156414
+
+ Reviewed by Oliver Hunt.
+
+ When size exceeds alignment and is a multiple of alignment and is not
+ a power of two, such as 24kB with 8kB alignment, the small allocator
+ did not always guarantee alignment. Let's fix that.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::divideRoundingUp): Math is hard.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocate): Align to the page size unconditionally.
+ Even if the page size is not a power of two, it might be a multiple of
+ a power of two, and we want alignment to that smaller power of two to
+ be guaranteed.
+
+2016-04-06 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: handle aligned allocations on the fast path
+ https://bugs.webkit.org/show_bug.cgi?id=156302
+
+ Reviewed by Michael Saboff.
+
+ This helps keep the JavaScriptCore GC on the fast path, and it also
+ helps avoid fragmentation on our website stress test:
+
+ nimlang 209,584kB 198,076kB ^ 1.06x smaller
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocate): Because we arrange for power-of-two size
+ classes to allocate at power-of-two alignments, we can allocate any
+ small aligned request on the small path.
+
+ * bmalloc/Chunk.h:
+ (bmalloc::Chunk::bytes):
+ (bmalloc::Chunk::lines):
+ (bmalloc::Chunk::pages):
+ (bmalloc::Chunk::boundaryTags):
+ (bmalloc::Chunk::objectType): Moved some code around to provide better
+ API.
+
+ (bmalloc::Chunk::Chunk): Moved this code to VMHeap.
+
+ (bmalloc::Chunk::offset):
+ (bmalloc::Chunk::object): Use our new bytes() helper function.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::allocateChunk): Moved code here from Chunk.
+
+ (bmalloc::VMHeap::allocateSmallChunk): Ensure that power-of-two page
+ sizes always begin allocation at the same alignment. Power-of-two object
+ sizes always request power-of-two page sizes (since that's the least
+ wasteful option), so if we also ensure that power-of-two page sizes get
+ power-of-two alignment, then everything is aligned for all small objects.
+
+2016-04-03 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: segregate small and large objects again, and allocate more objects on the small path
+ https://bugs.webkit.org/show_bug.cgi?id=156152
+
+ Reviewed by Sam Weinig.
+
+ Microbenchmark data suggested that it was a good idea for small and large
+ objects to share memory. But r198675 did not improve memory use in
+ full browser benchmarks.
+
+ This patch reverts to segregating small and large objects -- but without
+ going back to doubled VM usage -- in order to capture a few benefits:
+
+ (*) Small pages fragment the large heap. Separating them out saves a lot
+ of memory in our worst case fragmentation recording:
+
+ nimlang 276,076kB 209,636kB ^ 1.32x smaller
+
+ (*) Small objects are common enough that even their slow paths benefit
+ from simpler code:
+
+ Execution Time:
+ ...
+ facebook 234ms 216ms ^ 1.08x faster
+ reddit 114ms 108ms ^ 1.06x faster
+ flickr 118ms 111ms ^ 1.06x faster
+ theverge 146ms 140ms ^ 1.04x faster
+ ...
+ <arithmetic mean> 107ms 102ms ^ 1.04x faster
+
+ (*) We can use less metadata:
+
+ Memory at End:
+ ...
+ list_allocate 460kB 384kB ^ 1.2x smaller
+ tree_allocate 492kB 424kB ^ 1.16x smaller
+ tree_churn 480kB 404kB ^ 1.19x smaller
+ fragment 532kB 452kB ^ 1.18x smaller
+ fragment_iterate 712kB 588kB ^ 1.21x smaller
+ medium 15,152kB 11,796kB ^ 1.28x smaller
+ big 15,044kB 10,976kB ^ 1.37x smaller
+ ...
+ <arithmetic mean> 7,724kB 7,190kB ^ 1.07x smaller
+
+ This patch also takes advantage of our support for varying the page size
+ at runtime by allocating more objects on the small object path:
+
+ medium 178ms 150ms ^ 1.19x faster
+
+ Some microbenchmarks report memory use increases from this change -- like
+ they reported memory use decreases from r198675 -- but I'm ignoring them
+ for now because I expect our full browser memory benchmarks to confirm
+ that this patch is fine.
+
+ * bmalloc/BumpAllocator.h:
+ (bmalloc::BumpAllocator::BumpAllocator): Use a full unsigned because we
+ can allocate objects larger than 16kB - 1, and a full unsigned does not
+ make BumpAllocator any larger on 64bit systems.
+
+ * bmalloc/Chunk.h:
+ (bmalloc::Chunk::begin):
+ (bmalloc::Chunk::end):
+ (bmalloc::Chunk::size):
+ (bmalloc::Chunk::objectType): Store ObjectType in the Chunk, since it only
+ varies by Chunk now, and not from page to page within a Chunk. Also,
+ union together small and large object metadata, since we will only use
+ one or the other. This saves memory.
+
+ (bmalloc::Chunk::Chunk): Conditionalize initialization based on object
+ type, since only one kind of metadata or the other can be used at runtime.
+
+ (bmalloc::Object::Object):
+ (bmalloc::Object::begin):
+ (bmalloc::SmallPage::end): Deleted.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::initializeLineMetadata): Save a little space, since we
+ know that lines are only 256 bytes long.
+
+ (bmalloc::Heap::initializePageMetadata): Store a dynamic page size for
+ each size class. We used to use only one page size (the system page size)
+ but that limited our ability to allocate objects larger than 1kB on the
+ small object path. Now we can handle any object size we want by storing
+ objects of that size in a custom page size.
+
+ (bmalloc::Heap::concurrentScavenge):
+ (bmalloc::Heap::scavenge):
+ (bmalloc::Heap::scavengeSmallPages): Revert to our old linked list
+ strategy for storing small pages.
+
+ (bmalloc::Heap::splitAndAllocate): Object type is per Chunk now.
+
+ (bmalloc::Heap::allocateLarge): Don't nuke the small page list when
+ allocating a large object because the two don't share memory anymore.
+
+ (bmalloc::Heap::allocateSmallPage): Revert to our old linked list
+ strategy for storing small pages.
+
+ (bmalloc::Heap::deallocateSmallLine): Don't return early in the case
+ where this is the first free object in the page. In the case of large-ish
+ objects, the first free object might also be the last free object,
+ since there's one object per page.
+
+ (bmalloc::Heap::allocateSmallBumpRangesByMetadata): Split out some helper
+ lambdas to make this code clearer.
+
+ (bmalloc::Heap::allocateSmallBumpRangesByObject): Added a fast scan
+ for objects larger than the line size. When multiple objects fit in
+ a single line, it's an optimization to scan a line at a time. But when
+ it's one object per line, or one object per 64 lines, it's better just
+ to scan an object at a time.
+
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::allocateSmallBumpRanges):
+ (bmalloc::Heap::derefSmallLine): Match the changes above.
+
+ * bmalloc/LineMetadata.h: We weren't using all those bits.
+
+ * bmalloc/List.h:
+ (bmalloc::List::remove): Put a removed Node fully back into the default
+ (empty) state it was in before it entered the list. This change is not
+ observable, but it makes things clearer when you're debugging.
+
+ * bmalloc/Object.h:
+ (bmalloc::Object::Object):
+ (bmalloc::Object::chunk):
+ (bmalloc::Object::offset):
+ (bmalloc::Object::operator+):
+ (bmalloc::Object::operator<=): Added some helpers for iterating by object.
+
+ * bmalloc/ObjectType.cpp:
+ (bmalloc::objectType): Updated for API change.
+
+ * bmalloc/Sizes.h:
+ (bmalloc::Sizes::maskObjectSize):
+ (bmalloc::Sizes::objectSize):
+ (bmalloc::Sizes::pageSize): Support more page sizes.
+
+ * bmalloc/SmallPage.h:
+ (bmalloc::SmallPage::SmallPage):
+ (bmalloc::SmallPage::objectType): Deleted.
+ (bmalloc::SmallPage::setObjectType): Deleted.
+ (bmalloc::SmallPage::smallPageCount): Deleted.
+ (bmalloc::SmallPage::setSmallPageCount): Deleted. Object type is per
+ Chunk now, and we can infer page count from size class.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::allocateChunk):
+ (bmalloc::VMHeap::allocateSmallChunk):
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateSmallPage):
+ (bmalloc::VMHeap::deallocateSmallPage):
+ (bmalloc::VMHeap::allocateLargeObject): Support our old behavior of
+ storing free pages in linked lists.
+
+2016-03-29 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: support physical page sizes that don't match the virtual page size (take 2)
+ https://bugs.webkit.org/show_bug.cgi?id=156003
+
+ Reviewed by Andreas Kling.
+
+ This is a memory savings on iOS devices where the virtual page size
+ is 16kB but the physical page size is 4kB.
+
+ Take 1 was a memory regression on 16kB virtual / 16kB physical systems
+ because it used a 4kB page size within a 16kB page size, allowing up to
+ 4 different object types to mix within a physical page. Because objects
+ of the same type tend to deallocate at the same time, mixing objects of
+ different types made pages less likely to become completely empty.
+
+ (Take 1 also had a bug where it used a platform #ifdef that didn't exist.
+ Oops.)
+
+ Take 2 allocates units of SmallPages equal to the physical page size.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::initializeLineMetadata):
+ (bmalloc::Heap::allocateSmallBumpRanges):
+ (bmalloc::Heap::allocateSmallPage):
+ (bmalloc::Heap::allocateLarge):
+ (bmalloc::Heap::splitAndAllocate):
+ (bmalloc::Heap::tryAllocateXLarge):
+ (bmalloc::Heap::shrinkXLarge):
+ * bmalloc/Heap.h: Use the physical page size for our VM operations because
+ we're only concerned with returning physical pages to the OS.
+
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmPageSize):
+ (bmalloc::vmPageShift):
+ (bmalloc::vmSize):
+ (bmalloc::vmValidate):
+ (bmalloc::vmPageSizePhysical):
+ (bmalloc::vmValidatePhysical):
+ (bmalloc::tryVMAllocate):
+ (bmalloc::vmDeallocatePhysicalPages):
+ (bmalloc::vmAllocatePhysicalPages):
+ (bmalloc::vmDeallocatePhysicalPagesSloppy):
+ (bmalloc::vmAllocatePhysicalPagesSloppy): Use the physical page size.
+
+2016-03-29 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: page size should be configurable at runtime
+ https://bugs.webkit.org/show_bug.cgi?id=155993
+
+ Reviewed by Andreas Kling.
+
+ This is a memory win on 32bit iOS devices, since their page sizes are
+ 4kB and not 16kB.
+
+ It's also a step toward supporting 64bit iOS devices that have a
+ 16kB/4kB virtual/physical page size split.
+
+ * bmalloc/Chunk.h: Align to largeAlignment since 2 * smallMax isn't
+ required by the boundary tag allocator.
+
+ (bmalloc::Chunk::page): Account for the slide when accessing a page.
+ Each SmallPage hashes 4kB of memory. When we want to allocate a region
+ of memory larger than 4kB, we store our metadata in the first SmallPage
+ in the region and we assign a slide to the remaining SmallPages, so
+ they forward to that first SmallPage when accessed.
+
+ NOTE: We could use a less flexible technique that just hashed by
+ vmPageSize() instead of 4kB at runtime, with no slide, but I think we'll
+ be able to use this slide technique to make even more page sizes
+ dynamically at runtime, which should save some memory and simplify
+ the allocator.
+
+ (bmalloc::SmallPage::begin): It's invalid to access a SmallPage with
+ a slide, since such SmallPages do not contain meaningful data.
+
+ (bmalloc::SmallPage::end): Account for smallPageCount when computing
+ the size of a page.
+
+ (bmalloc::Chunk::pageBegin): Deleted.
+ (bmalloc::Chunk::pageEnd): Deleted.
+ (bmalloc::Object::pageBegin): Deleted.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap): Cache vmPageSize because computing it might require
+ a syscall.
+
+ (bmalloc::Heap::initializeLineMetadata): Line metadata is a vector instead
+ of a 2D array because we don't know how much metadata we'll need until
+ we know the page size.
+
+ (bmalloc::Heap::scavengeSmallPage): Be sure to revert the slide when
+ deallocating a page. Otherwise, the next attempt to allocate the page
+ will slide when initializing it, sliding to nowhere.
+
+ (bmalloc::Heap::allocateSmallBumpRanges): Account for vector change to
+ line metadata.
+
+ (bmalloc::Heap::allocateSmallPage): Initialize slide and smallPageCount
+ since they aren't constant anymore.
+
+ (bmalloc::Heap::allocateLarge):
+ (bmalloc::Heap::splitAndAllocate):
+ (bmalloc::Heap::tryAllocateXLarge):
+ (bmalloc::Heap::shrinkXLarge): Adopt dynamic page size.
+
+ * bmalloc/Heap.h:
+
+ * bmalloc/Sizes.h: smallPageSize is no longer equal to the VM page
+ size -- it's just the smallest VM page size we're interested in supporting.
+
+ * bmalloc/SmallPage.h:
+ (bmalloc::SmallPage::slide):
+ (bmalloc::SmallPage::setSlide):
+ (bmalloc::SmallPage::smallPageCount):
+ (bmalloc::SmallPage::setSmallPageCount):
+ (bmalloc::SmallPage::ref):
+ (bmalloc::SmallPage::deref): Support slide and small page count as
+ dynamic values. This doesn't increase metadata size since sizeof(SmallPage)
+ rounds up to alignment anyway.
+
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmPageSize):
+ (bmalloc::vmPageShift):
+ (bmalloc::vmSize):
+ (bmalloc::vmValidate):
+ (bmalloc::tryVMAllocate):
+ (bmalloc::vmDeallocatePhysicalPagesSloppy):
+ (bmalloc::vmAllocatePhysicalPagesSloppy): Treat page size as a variable.
+
+ * bmalloc/Vector.h:
+ (bmalloc::Vector::initialCapacity):
+ (bmalloc::Vector<T>::insert):
+ (bmalloc::Vector<T>::grow):
+ (bmalloc::Vector<T>::shrink):
+ (bmalloc::Vector<T>::shrinkCapacity):
+ (bmalloc::Vector<T>::growCapacity): Treat page size as a variable.
+
+2016-03-29 David Kilzer <ddkilzer@apple.com>
+
+ bmalloc: add logging for mmap() failures
+ <http://webkit.org/b/155409>
+ <rdar://problem/24568515>
+
+ Reviewed by Saam Barati.
+
+ This patch causes additional logging to be generated on internal
+ iOS builds when mmap() fails. We are trying to track down an
+ issue where the WebContent process runs out of VM address space
+ before it is killed by jetsam.
+
+ * CMakeLists.txt: Add Logging.cpp.
+ * bmalloc.xcodeproj/project.pbxproj: Add new files.
+
+ * bmalloc/BAssert.h:
+ (RELEASE_BASSERT_WITH_MESSAGE): Add macro.
+ * bmalloc/Logging.cpp: Added.
+ (bmalloc::logVMFailure): Implementation.
+ * bmalloc/Logging.h: Added.
+ (bmalloc::logVMFailure): Declaration.
+ * bmalloc/VMAllocate.h:
+ (bmalloc::tryVMAllocate): Call logVMFailure() on mmap() failure.
+ * bmalloc/darwin/BSoftLinking.h: Copied from Source/WebCore/platform/mac/SoftLinking.h.
+
+2016-03-26 Geoffrey Garen <ggaren@apple.com>
+
+ Unreviewed, rolling out r198702, r198704.
+
+ Caused a memory regression on PLUM.
+
+ Reverted changeset:
+
+ bmalloc: fix an ASSERT on iOS
+ https://bugs.webkit.org/show_bug.cgi?id=155911
+ http://trac.webkit.org/changeset/198704
+
+ bmalloc: support physical page sizes that don't match the virtual page size
+ https://bugs.webkit.org/show_bug.cgi?id=155898
+ http://trac.webkit.org/changeset/198702
+
+2016-03-25 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: fix an ASSERT on iOS
+ https://bugs.webkit.org/show_bug.cgi?id=155911
+
+ Reviewed by Gavin Barraclough.
+
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmValidatePhysical): Call through to vmValidatePhysical because
+ the vmValidate function validates virtual sizes rather than physical
+ sizes.
+
+2016-03-25 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: support physical page sizes that don't match the virtual page size
+ https://bugs.webkit.org/show_bug.cgi?id=155898
+
+ Reviewed by Gavin Barraclough.
+
+ This is a memory savings on iOS devices where the virtual page size
+ is 16kB but the physical page size is 4kB.
+
+ * bmalloc/Chunk.h:
+ (bmalloc::Chunk::Chunk): smallPageSize is now unrelated to the OS's
+ page size -- it just reflects the optimal unit of memory to recycle
+ between small objects.
+
+ We only need to round up to largeAlignment because small objects allocate
+ as subsets of large objects now.
+
+ (bmalloc::Chunk::page):
+ (bmalloc::Object::pageBegin):
+ (bmalloc::Object::line): Adopt smallPageSize.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::initializeLineMetadata):
+ (bmalloc::Heap::allocateSmallPage):
+ (bmalloc::Heap::allocateLarge): Adopt smallPageSize.
+
+ (bmalloc::Heap::splitAndAllocate):
+ (bmalloc::Heap::tryAllocateXLarge):
+ (bmalloc::Heap::shrinkXLarge): Adopt vmPageSizePhysical(). We want the
+ physical page size because that's the unit at which the hardware MMU
+ will recycle memory.
+
+ * bmalloc/Sizes.h: Adopt smallPageSize.
+
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmPageSizePhysical):
+ (bmalloc::vmPageSize): Distinguish between page size, which is the virtual
+ memory page size advertised by the OS, and physical page size, which the
+ true hardware page size.
+
+ (bmalloc::vmSize):
+ (bmalloc::vmValidate):
+ (bmalloc::vmValidatePhysical):
+ (bmalloc::tryVMAllocate):
+ (bmalloc::vmDeallocatePhysicalPages):
+ (bmalloc::vmAllocatePhysicalPages):
+ (bmalloc::vmDeallocatePhysicalPagesSloppy):
+ (bmalloc::vmAllocatePhysicalPagesSloppy): Adopt vmPageSize() and
+ vmPageSizePhyiscal().
+
+ * bmalloc/Vector.h:
+ (bmalloc::Vector::initialCapacity):
+ (bmalloc::Vector<T>::shrink):
+ (bmalloc::Vector<T>::shrinkCapacity):
+ (bmalloc::Vector<T>::growCapacity): Adopt vmPageSize(). We'd prefer to
+ use vmPageSizePhysical() but mmap() doesn't support it.
+
+ * bmalloc/XLargeMap.cpp: #include.
+
+2016-03-25 Geoffrey Garen <ggaren@apple.com>
+
+ Unreviewed, rolling in r198679.
+
+ r198679 was just a rename. The regression was caused by r198675 and then
+ fixed in r198693.
+
+ Restored changeset:
+
+ "bmalloc: Renamed LargeChunk => Chunk"
+ https://bugs.webkit.org/show_bug.cgi?id=155894
+ http://trac.webkit.org/changeset/198679
+
+2016-03-25 Geoffrey Garen <ggaren@apple.com>
+
+ Unreviewed, try to fix a crash seen on the bots.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate): We have to take the lock even if we're
+ only reading our own data becuse LargeObject contains validation code
+ that will read our neighbors' data as well.
+
+2016-03-25 Ryan Haddad <ryanhaddad@apple.com>
+
+ Unreviewed, rolling out r198679.
+
+ This change caused flaky LayoutTest crashes
+
+ Reverted changeset:
+
+ "bmalloc: Renamed LargeChunk => Chunk"
+ https://bugs.webkit.org/show_bug.cgi?id=155894
+ http://trac.webkit.org/changeset/198679
+
+2016-03-25 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: stress_aligned fails when allocating a zero-sized object with XLarge alignment
+ https://bugs.webkit.org/show_bug.cgi?id=155896
+
+ Reviewed by Andreas Kling.
+
+ We normally filter zero-sized allocations into small allocations, but
+ a zero-sized allocation can sneak through if it requires sufficiently
+ large alignment.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::tryAllocateXLarge): Set a floor on allocation size to
+ catch zero-sized allocations.
+
+2016-03-25 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Renamed LargeChunk => Chunk
+ https://bugs.webkit.org/show_bug.cgi?id=155894
+
+ Reviewed by Michael Saboff.
+
+ A Chunk can contain both small and large objects now.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocate):
+ * bmalloc/BoundaryTag.h:
+ (bmalloc::BoundaryTag::isFree):
+ * bmalloc/Chunk.h: Copied from Source/bmalloc/bmalloc/LargeChunk.h.
+ (bmalloc::Chunk::pages):
+ (bmalloc::Chunk::begin):
+ (bmalloc::Chunk::end):
+ (bmalloc::Chunk::Chunk):
+ (bmalloc::Chunk::get):
+ (bmalloc::Chunk::beginTag):
+ (bmalloc::Chunk::endTag):
+ (bmalloc::Chunk::offset):
+ (bmalloc::Chunk::object):
+ (bmalloc::Chunk::page):
+ (bmalloc::Chunk::line):
+ (bmalloc::SmallLine::begin):
+ (bmalloc::SmallPage::begin):
+ (bmalloc::SmallPage::end):
+ (bmalloc::Object::Object):
+ (bmalloc::Object::begin):
+ (bmalloc::LargeChunk::pages): Deleted.
+ (bmalloc::LargeChunk::begin): Deleted.
+ (bmalloc::LargeChunk::end): Deleted.
+ (bmalloc::LargeChunk::LargeChunk): Deleted.
+ (bmalloc::LargeChunk::get): Deleted.
+ (bmalloc::LargeChunk::beginTag): Deleted.
+ (bmalloc::LargeChunk::endTag): Deleted.
+ (bmalloc::LargeChunk::offset): Deleted.
+ (bmalloc::LargeChunk::object): Deleted.
+ (bmalloc::LargeChunk::page): Deleted.
+ (bmalloc::LargeChunk::line): Deleted.
+ * bmalloc/Deallocator.cpp:
+ * bmalloc/FreeList.cpp:
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateLarge):
+ * bmalloc/LargeChunk.h: Removed.
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::LargeObject):
+ (bmalloc::LargeObject::merge):
+ (bmalloc::LargeObject::split):
+ * bmalloc/Object.h:
+ (bmalloc::Object::chunk):
+ * bmalloc/ObjectType.cpp:
+ * bmalloc/Sizes.h:
+ * bmalloc/SmallAllocator.h: Removed.
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::VMHeap):
+ (bmalloc::VMHeap::allocateChunk):
+ (bmalloc::VMHeap::allocateLargeChunk): Deleted.
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateLargeObject):
+ (bmalloc::VMHeap::deallocateLargeObject):
+ * bmalloc/Zone.cpp:
+ (bmalloc::enumerator):
+ * bmalloc/Zone.h:
+ (bmalloc::Zone::chunks):
+ (bmalloc::Zone::addChunk):
+ (bmalloc::Zone::largeChunks): Deleted.
+ (bmalloc::Zone::addLargeChunk): Deleted.
+
+2016-03-24 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: small and large objects should share memory
+ https://bugs.webkit.org/show_bug.cgi?id=155866
+
+ Reviewed by Andreas Kling.
+
+ This patch cuts our VM footprint in half. (VM footprint usually doesn't
+ matter, but on iOS there's an artificial VM limit around 700MB, and if
+ you hit it you jetsam / crash.)
+
+ It's also a step toward honoring the hardware page size at runtime,
+ which will reduce memory usage on iOS.
+
+ This patch is a small improvement in peak memory usage because it allows
+ small and large objects to recycle each other's memory. The tradeoff is
+ that we require more metadata, which causes more memory usage after
+ shrinking down from peak memory usage. In the end, we have some memory
+ wins and some losses, and a small win in the mean on our standard memory
+ benchmarks.
+
+ * bmalloc.xcodeproj/project.pbxproj: Removed SuperChunk.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate): Adopt a new Heap API for shrinking
+ large objects because it's a little more complicated than it used to be.
+
+ Don't check for equality in the XLarge case because we don't do it in
+ other cases, and it's unlikely that we'll be called for no reason.
+
+ * bmalloc/BumpAllocator.h:
+ (bmalloc::BumpAllocator::allocate): Don't ASSERT isSmall because that's
+ an old concept from when small and large objects were in distinct memory
+ regions.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::deallocateSlowCase): Large objects are not
+ segregated anymore.
+
+ (bmalloc::Deallocator::deallocateLarge): Deleted.
+
+ * bmalloc/Deallocator.h:
+ (bmalloc::Deallocator::deallocateFastCase): Don't ASSERT isSmall(). See
+ above.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::scavenge):
+ (bmalloc::Heap::scavengeSmallPage):
+ (bmalloc::Heap::scavengeSmallPages): New helpers for returning cached
+ small pages to the large object heap.
+
+ (bmalloc::Heap::allocateSmallPage): Allocate small pages from the large
+ object heap. This is how we accomplish sharing.
+
+ (bmalloc::Heap::deallocateSmallLine): Handle large objects since we can
+ encounter them on this code path now.
+
+ (bmalloc::Heap::splitAndAllocate): Fixed a bug where we would sometimes
+ not split even though we could.
+
+ Allocating a large object also requires ref'ing its small line so that
+ we can alias memory between small and large objects.
+
+ (bmalloc::Heap::allocateLarge): Return cached small pages before
+ allocating a large object that would fit in a cached small page. This
+ allows some large allocations to reuse small object memory.
+
+ (bmalloc::Heap::shrinkLarge): New helper.
+
+ (bmalloc::Heap::deallocateLarge): Deleted.
+
+ * bmalloc/Heap.h:
+
+ * bmalloc/LargeChunk.h:
+ (bmalloc::LargeChunk::pageBegin):
+ (bmalloc::LargeChunk::pageEnd):
+ (bmalloc::LargeChunk::lines):
+ (bmalloc::LargeChunk::pages):
+ (bmalloc::LargeChunk::begin):
+ (bmalloc::LargeChunk::end):
+ (bmalloc::LargeChunk::LargeChunk):
+ (bmalloc::LargeChunk::get):
+ (bmalloc::LargeChunk::endTag):
+ (bmalloc::LargeChunk::offset):
+ (bmalloc::LargeChunk::object):
+ (bmalloc::LargeChunk::page):
+ (bmalloc::LargeChunk::line):
+ (bmalloc::SmallLine::begin):
+ (bmalloc::SmallLine::end):
+ (bmalloc::SmallPage::begin):
+ (bmalloc::SmallPage::end):
+ (bmalloc::Object::Object):
+ (bmalloc::Object::begin):
+ (bmalloc::Object::pageBegin):
+ (bmalloc::Object::line):
+ (bmalloc::Object::page): I merged all the SmallChunk metadata and code
+ into LargeChunk. Now we use a single class to track both small and large
+ metadata, so we can share memory between small and large objects.
+
+ I'm going to rename this class to Chunk in a follow-up patch.
+
+ * bmalloc/Object.h:
+ (bmalloc::Object::chunk): Updated for LargeChunk transition.
+
+ * bmalloc/ObjectType.cpp:
+ (bmalloc::objectType):
+ * bmalloc/ObjectType.h:
+ (bmalloc::isXLarge):
+ (bmalloc::isSmall): Deleted. The difference between small and large
+ objects is now stored in metadata and is not a property of their
+ virtual address range.
+
+ * bmalloc/SegregatedFreeList.h: One more entry because we cover all of
+ what used to be the super chunk in a large chunk now.
+
+ * bmalloc/Sizes.h: Removed bit masking helpers because we don't use
+ address masks to distinguish small vs large object type anymore.
+
+ * bmalloc/SmallChunk.h: Removed.
+
+ * bmalloc/SmallPage.h:
+ (bmalloc::SmallPage::SmallPage): Store object type per page because any
+ given page can be used for large objects or small objects.
+
+ * bmalloc/SuperChunk.h: Removed.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::VMHeap):
+ (bmalloc::VMHeap::allocateLargeChunk):
+ (bmalloc::VMHeap::allocateSmallChunk): Deleted.
+ (bmalloc::VMHeap::allocateSuperChunk): Deleted.
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateLargeObject):
+ (bmalloc::VMHeap::deallocateLargeObject):
+ (bmalloc::VMHeap::allocateSmallPage): Deleted.
+ (bmalloc::VMHeap::deallocateSmallPage): Deleted. Removed super chunk and
+ small chunk support.
+
+ * bmalloc/Zone.cpp:
+ (bmalloc::enumerator):
+ * bmalloc/Zone.h:
+ (bmalloc::Zone::largeChunks):
+ (bmalloc::Zone::addLargeChunk):
+ (bmalloc::Zone::superChunks): Deleted.
+ (bmalloc::Zone::addSuperChunk): Deleted. Removed super chunk and
+ small chunk support.
+
+2016-03-23 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Added an Object helper class
+ https://bugs.webkit.org/show_bug.cgi?id=155818
+
+ Reviewed by Gavin Barraclough.
+
+ Object is an abstraction that breaks out a void* into its component
+ metadata pointers.
+
+ This is slightly faster than recomputing them, and it enables a future
+ patch in which Object will tell us whether it is small or large.
+
+ * bmalloc.xcodeproj/project.pbxproj: Added to the project.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate): Use Object to compute size.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::processObjectLog):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateSmallPage):
+ (bmalloc::Heap::deallocateSmallLine):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::derefSmallLine): Use Object to deallocate.
+
+ * bmalloc/Object.h: Added.
+ (bmalloc::Object::Object):
+ (bmalloc::Object::chunk):
+ (bmalloc::Object::line):
+ (bmalloc::Object::page): Helper class to break out a void* into its
+ component metadata pointers.
+
+ * bmalloc/SmallChunk.h:
+ (bmalloc::SmallChunk::SmallChunk): SmallPage::get doesn't exist anymore
+ so we use our new helper functions instead.
+
+ (bmalloc::SmallChunk::offset):
+ (bmalloc::SmallChunk::object):
+ (bmalloc::SmallChunk::page):
+ (bmalloc::SmallChunk::line):
+ (bmalloc::SmallLine::begin):
+ (bmalloc::SmallLine::end):
+ (bmalloc::SmallPage::begin): New helpers that operate on the data
+ stored in Object.
+
+ (bmalloc::SmallLine::get): Deleted.
+ (bmalloc::SmallPage::get): Deleted.
+
+ * bmalloc/SmallLine.h:
+ (bmalloc::SmallLine::refCount): Added a default ref value for convenience.
+
+ * bmalloc/SmallPage.h:
+ (bmalloc::SmallPage::SmallPage):
+
+2016-03-23 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: process the object log before asking for new memory
+ https://bugs.webkit.org/show_bug.cgi?id=155801
+
+ Reviewed by Gavin Barraclough.
+
+ This is a step toward merging large and small objects: In future, if we
+ have large objects in the log, we need to process them right away to
+ avoid pushing up peak memory use.
+
+ But it also appears to be a speedup and memory use improvement now.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocate):
+ (bmalloc::Allocator::refillAllocatorSlowCase):
+ (bmalloc::Allocator::allocateLarge): Process the log before asking for
+ more memory.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::processObjectLog):
+ (bmalloc::Deallocator::deallocateSlowCase):
+ * bmalloc/Deallocator.h: Provide a public API for processing the object log.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateSmallPage): Pop fragmented pages from the front
+ instead of from the back. This resolves a regression on tree_churn
+ --parallel. Popping from the front gives us the oldest pages. The oldest
+ pages have had the most time to accumulate free lines. They are therefore
+ the least fragmented on average.
+
+ * bmalloc/List.h:
+ (bmalloc::List::popFront):
+ (bmalloc::List::insertAfter): New API to pop from front.
+
+2016-03-22 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: use a log scale for large-ish size classes
+ https://bugs.webkit.org/show_bug.cgi?id=155770
+
+ Reviewed by Michael Saboff.
+
+ At larger sizes, precise allocation sizes don't save much memory -- and
+ they can cost memory when objects of distinct size classes can't
+ allocate together.
+
+ This is a small savings up to our current allocation limits, and it may
+ enable changing those limits in the long term.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::log2): We use this to compute large-ish size classes.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::Allocator): Iterate by size class instead of by
+ object size so we can change object size limits without breaking stuff.
+
+ (bmalloc::Allocator::scavenge): Ditto.
+
+ (bmalloc::Allocator::allocateLogSizeClass): New helper function for
+ allocating based on log size classes.
+
+ (bmalloc::Allocator::allocateSlowCase): Account for extra size class
+ possibilities.
+
+ * bmalloc/Allocator.h:
+ (bmalloc::Allocator::allocateFastCase): We only handle up to 512b on
+ the fastest fast path now.
+
+ * bmalloc/BumpAllocator.h:
+ (bmalloc::BumpAllocator::validate): Deleted. I noticed that this function
+ had been refactored not to do anything anymore.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::initializeLineMetadata): Iterate by size class. (See
+ Allocator::Allocator.)
+
+ * bmalloc/Heap.h: Use the sizeClassCount constant instead of hard coding
+ things.
+
+ * bmalloc/Sizes.h:
+ (bmalloc::Sizes::maskSizeClass):
+ (bmalloc::Sizes::maskObjectSize):
+ (bmalloc::Sizes::logSizeClass):
+ (bmalloc::Sizes::logObjectSize):
+ (bmalloc::Sizes::sizeClass):
+ (bmalloc::Sizes::objectSize): Separate size class calculation between
+ simple size classes that can be computed with a mask and are 8-byte-precise
+ and complex size classes that require more math and are less precise.
+
+ * bmalloc/SmallLine.h:
+ (bmalloc::SmallLine::ref):
+ * bmalloc/SmallPage.h:
+ (bmalloc::SmallPage::SmallPage):
+ (bmalloc::SmallPage::ref):
+ (bmalloc::SmallPage::deref): Cleaned up some ASSERTs that triggered
+ while working on this patch.
+
+ * bmalloc/Zone.cpp:
+ (bmalloc::statistics):
+ (bmalloc::zoneSize):
+ (bmalloc::Zone::Zone):
+ (bmalloc::size): Deleted. Renamed these symbols to work around an lldb
+ bug that makes it impossible to print out variables named 'size' -- which
+ can be a problem when working on malloc.
+
+2016-03-22 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: shrink largeMax
+ https://bugs.webkit.org/show_bug.cgi?id=155759
+
+ Reviewed by Michael Saboff.
+
+ If a largeChunk contains N bytes and we allocate objects of size
+ N / 2 + 8 bytes, then we waste 50% of physical memory at peak.
+
+ This patch sets largeMax to N / 2, reducing maximum waste to 25%.
+
+ * bmalloc/BoundaryTag.h:
+ * bmalloc/LargeChunk.h:
+ (bmalloc::LargeChunk::LargeChunk):
+ * bmalloc/SegregatedFreeList.cpp:
+ (bmalloc::SegregatedFreeList::SegregatedFreeList):
+ (bmalloc::SegregatedFreeList::insert): Honor largeMax vs largeObjectMax.
+
+ * bmalloc/Sizes.h: Distinguish between the largest thing we can store
+ in a free list (largeObjectMax) and the largest thing we're willing to
+ allocate (largeMax).
+
+2016-03-20 Dan Bernstein <mitz@apple.com>
+
+ [Mac] Determine TARGET_MAC_OS_X_VERSION_MAJOR from MACOSX_DEPLOYMENT_TARGET rather than from MAC_OS_X_VERSION_MAJOR
+ https://bugs.webkit.org/show_bug.cgi?id=155707
+ <rdar://problem/24980691>
+
+ Reviewed by Darin Adler.
+
+ * Configurations/Base.xcconfig: Set TARGET_MAC_OS_X_VERSION_MAJOR based on the last
+ component of MACOSX_DEPLOYMENT_TARGET.
+ * Configurations/DebugRelease.xcconfig: For engineering builds, preserve the behavior of
+ TARGET_MAC_OS_X_VERSION_MAJOR being the host’s OS version.
+
+2016-03-20 Dan Bernstein <mitz@apple.com>
+
+ Update build settings
+
+ Rubber-stamped by Andy Estes.
+
+ * Configurations/DebugRelease.xcconfig:
+
+2016-03-14 Geoffrey Garen <ggaren@apple.com>
+
+ Unreviewed, rolling out r197955.
+
+ I decided to go in another direction
+
+ Reverted changeset:
+
+ "bmalloc: Rename SmallPage to SmallRun"
+ https://bugs.webkit.org/show_bug.cgi?id=155320
+ http://trac.webkit.org/changeset/197955
+
+2016-03-10 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Rename SmallPage to SmallRun
+ https://bugs.webkit.org/show_bug.cgi?id=155320
+
+ Reviewed by Alex Christensen.
+
+ A page is a fixed-size set of lines.
+
+ A run is an variable-sized set of lines.
+
+ We want to start using runs because:
+
+ (a) we want to support varying the hardware page size by OS;
+
+ (b) we want to support allocations larger than our current page size.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::initializeSmallRunMetadata):
+ (bmalloc::Heap::scavenge):
+ (bmalloc::Heap::scavengeSmallRuns):
+ (bmalloc::Heap::allocateSmallBumpRanges):
+ (bmalloc::Heap::allocateSmallRun):
+ (bmalloc::Heap::deallocateSmallLine):
+ (bmalloc::Heap::initializeLineMetadata): Deleted.
+ (bmalloc::Heap::scavengeSmallPages): Deleted.
+ (bmalloc::Heap::allocateSmallPage): Deleted.
+ * bmalloc/Heap.h:
+ * bmalloc/LineMetadata.h:
+ * bmalloc/SmallChunk.h:
+ (bmalloc::SmallChunk::begin):
+ (bmalloc::SmallChunk::end):
+ (bmalloc::SmallChunk::lines):
+ (bmalloc::SmallChunk::runs):
+ (bmalloc::SmallChunk::SmallChunk):
+ (bmalloc::SmallLine::end):
+ (bmalloc::SmallRun::get):
+ (bmalloc::SmallRun::begin):
+ (bmalloc::SmallRun::end):
+ (bmalloc::SmallChunk::pages): Deleted.
+ (bmalloc::SmallPage::get): Deleted.
+ (bmalloc::SmallPage::begin): Deleted.
+ (bmalloc::SmallPage::end): Deleted.
+ * bmalloc/SmallPage.h: Removed.
+ * bmalloc/SmallRun.h: Copied from Source/bmalloc/bmalloc/SmallPage.h.
+ (bmalloc::SmallRun::SmallRun):
+ (bmalloc::SmallRun::ref):
+ (bmalloc::SmallRun::deref):
+ (bmalloc::SmallPage::SmallPage): Deleted.
+ (bmalloc::SmallPage::ref): Deleted.
+ (bmalloc::SmallPage::deref): Deleted.
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::allocateSmallChunk):
+ (bmalloc::VMHeap::allocateLargeChunk):
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateSmallRun):
+ (bmalloc::VMHeap::allocateLargeObject):
+ (bmalloc::VMHeap::deallocateSmallRun):
+ (bmalloc::VMHeap::deallocateLargeObject):
+ (bmalloc::VMHeap::allocateSmallPage): Deleted.
+ (bmalloc::VMHeap::deallocateSmallPage): Deleted.
+
+2016-03-08 Geoffrey Garen <ggaren@apple.com>
+
+ Unreviewed, rolling in r197722.
+ https://bugs.webkit.org/show_bug.cgi?id=155171
+
+ The right calculation for our static_assert is actually:
+
+ sizeof(SmallChunk) % vmPageSize + 2 * smallMax <= vmPageSize
+
+ instead of:
+
+ sizeof(SmallChunk) % vmPageSize + smallMax <= vmPageSize
+
+ smallMax is not enough because line metadata might require us to begin
+ allocation at an offset as large as smallMax, so we need 2 * smallMax.
+
+ Once correct, this static_assert fires, and we fix it by increasing
+ the alignment of SmallChunk.
+
+ Restored changeset:
+
+ "bmalloc: Use List<T> instead of Vector<T> in some places"
+ https://bugs.webkit.org/show_bug.cgi?id=155150
+ http://trac.webkit.org/changeset/197722
+
+2016-03-08 Commit Queue <commit-queue@webkit.org>
+
+ Unreviewed, rolling out r197722.
+ https://bugs.webkit.org/show_bug.cgi?id=155171
+
+ This change caused 800+ JSC test failures (Requested by
+ ryanhaddad on #webkit).
+
+ Reverted changeset:
+
+ "bmalloc: Use List<T> instead of Vector<T> in some places"
+ https://bugs.webkit.org/show_bug.cgi?id=155150
+ http://trac.webkit.org/changeset/197722
+
+2016-03-07 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Use List<T> instead of Vector<T> in some places
+ https://bugs.webkit.org/show_bug.cgi?id=155150
+
+ Reviewed by Andreas Kling.
+
+ Vector<T> is expensive when you want a lot of them because our minimum
+ allocation size is the system page size.
+
+ * bmalloc.xcodeproj/project.pbxproj: Added a List<T> class.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::scavengeSmallPages):
+ (bmalloc::Heap::allocateSmallPage): Use the List<T> API. No need to check
+ for stale entries anymore because List<T> supports O(1) eager removal
+ and we remove eagerly now.
+
+ (bmalloc::Heap::deallocateSmallLine): Remove eagerly. This simplifies
+ the allocation code and it is also required for correctness since we
+ only have enough metadata to be in one list at a time.
+
+ * bmalloc/Heap.h: List!
+
+ * bmalloc/SmallChunk.h: Made this assert a little more precise since this
+ patch triggered the old version in a benign way.
+
+ (bmalloc::SmallChunk::SmallChunk): This code moved to the SmallPage
+ constructor.
+
+ * bmalloc/SmallPage.h:
+ (bmalloc::SmallPage::SmallPage): Accomodate the List<T> data structure.
+ This is a net memory savings on Mac for heaps smaller than ~128MB and on
+ iOS for heaps smaller than ~512MB. The maximum memory saved is 512kB on
+ Mac and 2MB on iOS. For larger heaps, there's a memory cost of 0.4% on
+ Mac and 0.1% on iOS.
+
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateSmallPage): Use List<T> API.
+
+2016-03-03 Geoffrey Garen <ggaren@apple.com>
+
+ Unreviewed, rolling in r197174.
+ https://bugs.webkit.org/show_bug.cgi?id=154762
+
+ The right calculation for alignment is actually:
+
+ vmAlignment - getpagesize() + vmSize
+
+ instead of:
+
+ vmAlignment - vmPageSize + vmSize
+
+ The vmPageSize might be larger than getpagesize().
+
+ Restored changeset:
+
+ "bmalloc: Added a fast XLarge allocator"
+ https://bugs.webkit.org/show_bug.cgi?id=154720
+ http://trac.webkit.org/changeset/197174
+
+2016-02-26 Commit Queue <commit-queue@webkit.org>
+
+ Unreviewed, rolling out r197174.
+ https://bugs.webkit.org/show_bug.cgi?id=154762
+
+ This change caused LayoutTests to crash on iOS simulator
+ (Requested by ryanhaddad on #webkit).
+
+ Reverted changeset:
+
+ "bmalloc: Added a fast XLarge allocator"
+ https://bugs.webkit.org/show_bug.cgi?id=154720
+ http://trac.webkit.org/changeset/197174
+
+2016-02-25 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Added a fast XLarge allocator
+ https://bugs.webkit.org/show_bug.cgi?id=154720
+
+ Reviewed by Andreas Kling.
+
+ This is a big speedup for XLarge allocations because it avoids mmap
+ and page fault churn. It also enables future design changes to handle
+ a smaller size range on the fast path.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::roundUpToMultipleOf):
+ (bmalloc::roundDownToMultipleOf): Added a non-constant round down.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::tryAllocate): XLarge no longer requires the caller
+ to align things.
+
+ (bmalloc::Allocator::allocate): Tweaked the alignment calculation for
+ clarity. When alignment and largeAlignment are equal, no adjustment
+ is necessary since all allocations guarantee largeAlignment.
+
+ (bmalloc::Allocator::reallocate): Updated for interface change.
+
+ Note that the new interface fixes some concurrency bugs. The old code
+ kept an iterator into the XLarge allocator across lock drop and acquisition,
+ which is not cool.
+
+ (bmalloc::Allocator::allocateXLarge): XLarge no longer requires the caller
+ to align things.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::scavengeXLargeObjects): Added scavenging for XLarge.
+
+ (bmalloc::Heap::allocateXLarge):
+
+ (bmalloc::Heap::splitAndAllocate): Split XLarge objects to xLargeAlignment.
+
+ (bmalloc::Heap::tryAllocateXLarge):
+ (bmalloc::Heap::xLargeSize):
+ (bmalloc::Heap::shrinkXLarge):
+ (bmalloc::Heap::deallocateXLarge): Allocate from our map before going
+ to the OS.
+
+ (bmalloc::Heap::findXLarge): Deleted.
+
+ * bmalloc/Heap.h:
+
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::split):
+
+ * bmalloc/ObjectType.h:
+ (bmalloc::isXLarge): Give XLarge objects an explicit alignment for clarity.
+
+ * bmalloc/Range.h:
+ (bmalloc::Range::size):
+ (bmalloc::Range::operator!):
+ (bmalloc::Range::operator bool):
+ (bmalloc::Range::operator<):
+ (bmalloc::canMerge):
+ (bmalloc::merge): Some helpers that were useful in writing this patch.
+
+ * bmalloc/Sizes.h:
+
+ * bmalloc/SortedVector.h: Added.
+ (bmalloc::SortedVector::Bucket::Bucket):
+ (bmalloc::SortedVector::Bucket::operator<):
+ (bmalloc::SortedVector::iterator::iterator):
+ (bmalloc::SortedVector::iterator::operator++):
+ (bmalloc::SortedVector::iterator::operator!=):
+ (bmalloc::SortedVector::iterator::operator*):
+ (bmalloc::SortedVector::iterator::operator->):
+ (bmalloc::SortedVector::iterator::skipDeletedBuckets):
+ (bmalloc::SortedVector::begin):
+ (bmalloc::SortedVector::end):
+ (bmalloc::SortedVector<T>::insert):
+ (bmalloc::SortedVector<T>::find):
+ (bmalloc::SortedVector<T>::get):
+ (bmalloc::SortedVector<T>::take):
+ (bmalloc::SortedVector<T>::shrinkToFit): A simple abstraction for keeping
+ a sorted vector. Insertion is average amortized log(n) because we keep
+ deleted buckets that we can reuse.
+
+ This is better than a tree because we get better locality, less memory
+ use, and simpler code. Also, trees require a node memory allocator, and
+ implementing a memory allocator in a memory allocator is no fun.
+
+ Arguably we should use a hash table instead. But that's more code, and
+ sorted vector has other nice properties that we might want to take
+ adavantage of in the future.
+
+ * bmalloc/VMAllocate.h:
+ (bmalloc::tryVMAllocate): Fixed an inaccuracy in the alignment calculation
+ here. This code was sort of trying to enforce the alignment that the
+ XLarge allocator enforces -- but it's better to enforce that alignment
+ there.
+
+ The right calculation is:
+
+ vmAlignment - vmPageSize + vmSize
+
+ because the worst case is when you are aligned to 0 + vmPageSize, and
+ you must walk forward vmAlignment - vmPageSize to reach the next
+ vmAlignment.
+
+ (bmalloc::tryVMExtend): Deleted. No need to go back to the OS for VM
+ since we manage our own.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::allocateLargeChunk): Updated for clarity. When we
+ grow the large heap we know that grown region is where the next allocation
+ will take place, so we return it directly instead of pushing it to the
+ free list.
+
+ This fixes a subtle bug where an overly conservative aligned allocation
+ algorithm can fail to allocate at all when it grows the heap.
+
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateLargeObject): Ditto.
+ (bmalloc::VMHeap::allocateLargeObject): Ditto.
+
+ * bmalloc/VMState.h:
+ (bmalloc::merge): Added a helper.
+
+ * bmalloc/Vector.h:
+ (bmalloc::Vector::begin):
+ (bmalloc::Vector::end):
+ (bmalloc::Vector::size):
+ (bmalloc::Vector::capacity):
+ (bmalloc::Vector::last):
+ (bmalloc::Vector::pop):
+ (bmalloc::Vector<T>::push):
+ (bmalloc::Vector<T>::pop):
+ (bmalloc::Vector<T>::shrink): Use a proper iterator API to play nice
+ with std algorithms.
+
+ (bmalloc::Vector<T>::insert): New function required by SortedVector.
+
+ (bmalloc::Vector<T>::reallocateBuffer):
+ (bmalloc::Vector<T>::shrinkCapacity): Allow for shrinking back all the way
+ to 0 because that's what shrinkToFit wants.
+ (bmalloc::Vector<T>::growCapacity):
+ (bmalloc::Vector<T>::shrinkToFit):
+
+ * bmalloc/XLargeMap.cpp: Added. Helper data structure for managing XLarge
+ objects. We have enough granularity in our metadata to represent any
+ kind of address range.
+
+ We store free ranges in a flat vector because most programs have very
+ few individual free XLarge ranges. (They usually merge.)
+
+ We store allocated ranges in a sorted vector because programs might
+ allocate lots of XLarge ranges. For example, if the XLarge minimum is
+ 128kB, and you have a 1GB process, that's 8192 ranges. Linear scan would
+ examine 8192 items but binary search only 13.
+
+ Empirically, this is 1.5X faster than our current large allocator if you
+ modify MallocBench/big to allocate XLarge objects and not to initialize
+ objects and you allocate 128kB-256kB objects in a 1GB address space.
+
+ (bmalloc::XLargeMap::takeFree): Be careful about overflow in this function
+ because we support super huge pointers, alignments, and sizes.
+
+ (bmalloc::XLargeMap::addFree): Merge eagerly on free because the cost
+ of missing an XLarge opportunity is catastrophic. Also, I discovered
+ by experiment that any allocator that doesn't merge eagerly can create
+ lots of subtle opportunities for snowballing fragmentation, as
+ fragmentation in range A forces you to chop up range B, and so on.
+
+ We allocate "first fit" (allocating the lowest address) because someone
+ wrote a paper once that said that it's the best algorithm to combat
+ fragmentation (even though worst case fragmentation is unavoidable
+ regardless of algorithm).
+
+ (bmalloc::XLargeMap::addAllocated):
+ (bmalloc::XLargeMap::getAllocated):
+ (bmalloc::XLargeMap::takeAllocated):
+ (bmalloc::XLargeMap::shrinkToFit):
+ (bmalloc::XLargeMap::takePhysical):
+ (bmalloc::XLargeMap::addVirtual):
+ * bmalloc/XLargeMap.h: Added.
+ (bmalloc::XLargeMap::Allocation::operator<):
+
+ * bmalloc/XLargeRange.h: Added.
+ (bmalloc::XLargeRange::XLargeRange):
+ (bmalloc::XLargeRange::vmState):
+ (bmalloc::XLargeRange::setVMState):
+ (bmalloc::canMerge):
+ (bmalloc::merge):
+ (bmalloc::XLargeRange::split): Helper for tracking VMState in a range.
+
+2016-02-23 Dan Bernstein <mitz@apple.com>
+
+ [Xcode] Linker errors display mangled names, but no longer should
+ https://bugs.webkit.org/show_bug.cgi?id=154632
+
+ Reviewed by Sam Weinig.
+
+ * Configurations/Base.xcconfig: Stop setting LINKER_DISPLAYS_MANGLED_NAMES to YES.
+
+2016-02-22 Konstantin Tokarev <annulen@yandex.ru>
+
+ Fixed compilation of bmalloc with GCC 4.8 after r196873.
+ https://bugs.webkit.org/show_bug.cgi?id=154534
+
+ Reviewed by Mark Lam.
+
+ See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=55382.
+
+ * bmalloc/LargeChunk.h:
+ * bmalloc/SmallChunk.h:
+
+2016-02-21 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Don't use a whole page for metadata
+ https://bugs.webkit.org/show_bug.cgi?id=154510
+
+ Reviewed by Andreas Kling.
+
+ (1) Don't round up metadata to a page boundary. This saves 1.5% dirty
+ memory on iOS and 0.2% on Mac. It also enables a future patch to allocate
+ smaller chunks without wasting memory.
+
+ (2) Initialize metadata lazily. This saves dirty memory when the program
+ allocates primarily small or large objects (but not both), leaving some
+ metadata uninitialized.
+
+ * bmalloc.xcodeproj/project.pbxproj: Medium objects are gone now.
+
+ * bmalloc/BumpAllocator.h:
+ (bmalloc::BumpAllocator::refill): Added an ASSERT to help debug a bug
+ I cause while working on this patch.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateSmallBumpRanges): Ditto.
+
+ (bmalloc::Heap::splitAndAllocate):
+ (bmalloc::Heap::allocateLarge): Updated for interface change.
+
+ * bmalloc/LargeChunk.h: Changed the boundaryTagCount calculation to
+ a static_assert.
+
+ Don't round up to page boundary. (See above.)
+
+ (bmalloc::LargeChunk::LargeChunk): Moved code here from LargeChunk::init.
+ A constructor is a more natural / automatic way to do this initialization.
+
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::init): Deleted. Moved to LargeChunk.
+
+ * bmalloc/Sizes.h: Chagned largeChunkMetadataSize to a simpler constant
+ because metadata size no longer varies by page size.
+
+ * bmalloc/SmallChunk.h:
+ (bmalloc::SmallChunk::begin):
+ (bmalloc::SmallChunk::end):
+ (bmalloc::SmallChunk::lines):
+ (bmalloc::SmallChunk::pages): Use std::array to make begin/end
+ calculations easier.
+
+ (bmalloc::SmallChunk::SmallChunk): Treat our metadata like a series
+ of allocated objects. We used to avoid trampling our metadata by
+ starting object memory at the next page. Now we share the first page
+ between metadata and objects, and we account for metadata explicitly.
+
+ * bmalloc/SuperChunk.h:
+ (bmalloc::SuperChunk::SuperChunk):
+ (bmalloc::SuperChunk::smallChunk):
+ (bmalloc::SuperChunk::largeChunk):
+ (bmalloc::SuperChunk::create): Deleted. Don't eagerly run the SmallChunk
+ and LargeChunk constructors. We'll run them lazily as needed.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::VMHeap):
+ (bmalloc::VMHeap::allocateSmallChunk):
+ (bmalloc::VMHeap::allocateLargeChunk):
+ (bmalloc::VMHeap::allocateSuperChunk):
+ (bmalloc::VMHeap::grow): Deleted. Track small and large chunks explicitly
+ so we can initialize them lazily.
+
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateSmallPage):
+ (bmalloc::VMHeap::allocateLargeObject): Specify whether we're allocating
+ a small or large chunk since we don't allocate both at once anymore.
+
+2016-02-20 Mark Lam <mark.lam@apple.com>
+
+ Use of inlined asm statements causes problems for -std=c99 builds.
+ https://bugs.webkit.org/show_bug.cgi?id=154507
+
+ Reviewed by Dan Bernstein.
+
+ * bmalloc/BAssert.h:
+
+2016-02-19 Joonghun Park <jh718.park@samsung.com>
+
+ Unreviewed. Fix debug build error since r196847
+
+ Fix gcc build warning appeared as below
+ by removing BASSERT(refCount <= maxRefCount).
+ error: comparison is always true due to limited range of data type
+ [-Werror=type-limits]
+
+ * bmalloc/SmallLine.h:
+ (bmalloc::SmallLine::ref): Deleted.
+
+2016-02-19 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Chunk, Page, and Line don't need to be class templates
+ https://bugs.webkit.org/show_bug.cgi?id=154480
+
+ Reviewed by Gavin Barraclough.
+
+ We needed class templates to distinguish between small and medium,
+ but medium is gone now.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Chunk.h: Removed.
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::initializeLineMetadata):
+ (bmalloc::Heap::allocateSmallBumpRanges):
+ * bmalloc/Heap.h:
+ * bmalloc/Line.h: Removed.
+ * bmalloc/Page.h: Removed.
+ * bmalloc/Sizes.h:
+ * bmalloc/SmallChunk.h: Replaced with Source/bmalloc/bmalloc/Chunk.h.
+ (bmalloc::SmallChunk::begin):
+ (bmalloc::SmallChunk::end):
+ (bmalloc::SmallChunk::lines):
+ (bmalloc::SmallChunk::pages):
+ (bmalloc::SmallChunk::get):
+ (bmalloc::SmallLine::get):
+ (bmalloc::SmallLine::begin):
+ (bmalloc::SmallLine::end):
+ (bmalloc::SmallPage::get):
+ (bmalloc::SmallPage::begin):
+ (bmalloc::SmallPage::end):
+ (bmalloc::Chunk::begin): Deleted.
+ (bmalloc::Chunk::end): Deleted.
+ (bmalloc::Chunk::lines): Deleted.
+ (bmalloc::Chunk::pages): Deleted.
+ * bmalloc/SmallLine.h: Replaced with Source/bmalloc/bmalloc/Line.h.
+ (bmalloc::SmallLine::ref):
+ (bmalloc::SmallLine::deref):
+ (bmalloc::Line<Traits>::begin): Deleted.
+ (bmalloc::Line<Traits>::end): Deleted.
+ (bmalloc::Line<Traits>::ref): Deleted.
+ (bmalloc::Line<Traits>::deref): Deleted.
+ * bmalloc/SmallPage.h: Replaced with Source/bmalloc/bmalloc/Page.h.
+ (bmalloc::SmallPage::hasFreeLines):
+ (bmalloc::SmallPage::setHasFreeLines):
+ (bmalloc::SmallPage::ref):
+ (bmalloc::SmallPage::deref):
+ (bmalloc::Page::hasFreeLines): Deleted.
+ (bmalloc::Page::setHasFreeLines): Deleted.
+ (bmalloc::Page<Traits>::ref): Deleted.
+ (bmalloc::Page<Traits>::deref): Deleted.
+ * bmalloc/SmallTraits.h: Removed.
+
+2016-02-18 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Remove the concept of medium objects
+ https://bugs.webkit.org/show_bug.cgi?id=154436
+
+ Reviewed by Sam Weinig.
+
+ There's no need to distinguish medium objects from small: Small object
+ metadata works naturally for both as long as we allow an object to
+ span more than two small lines. (We already allow an object to span
+ more than one small line.)
+
+ This change reduces memory use because it eliminates the 1kB line size,
+ so we don't have to hold down 1kB lines for individual 264+ byte objects.
+
+ 1kB lines were always a bit of a compromise. The main point of bump
+ allocation is to take advantage of cache lines. Cache lines are usually
+ 64 bytes, so line sizes above 256 bytes are a bit of a stretch.
+
+ This change speeds up small object benchmarks because it eliminates the
+ branch to detect medium objects in deallocation log processing.
+
+ This change reduces virtual memory use from worst cast 4X to worst case
+ 2X because the medium chunk is gone. iOS cares about virtual memory use
+ and terminates apps above ~1GB, so this change gives us more breathing room.
+
+ This change slows down medium benchmarks a bit because we end up doing
+ more work to recycle fragmented medium objects. Overall, the tradeoff
+ seems justified, since we have a net speedup and a memory use savings.
+
+ * bmalloc.xcodeproj/project.pbxproj: Removed all the medium files. We
+ can simplify even further in a follow-up patch, removing the base class
+ templates for Chunk, Page, and Line as well.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::Allocator):
+ (bmalloc::Allocator::allocate):
+ (bmalloc::Allocator::reallocate):
+ (bmalloc::Allocator::scavenge):
+ (bmalloc::Allocator::refillAllocatorSlowCase):
+ (bmalloc::Allocator::refillAllocator):
+ (bmalloc::Allocator::allocateSlowCase): Medium is gone. Small max is the
+ new medium max.
+
+ * bmalloc/Allocator.h:
+ (bmalloc::Allocator::allocateFastCase): Ditto.
+
+ * bmalloc/BumpAllocator.h:
+ (bmalloc::BumpAllocator::validate):
+ (bmalloc::BumpAllocator::allocate): No more medium.
+
+ * bmalloc/Chunk.h: No more medium.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::processObjectLog): No check for medium. This is
+ a speedup.
+
+ (bmalloc::Deallocator::deallocateSlowCase): No more medium.
+
+ * bmalloc/Deallocator.h:
+ (bmalloc::Deallocator::deallocateFastCase): Ditto.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::initializeLineMetadata): The algorithm here changed from
+ iterating each line to iterating each object. This helps us accomodate
+ objects that might span more than two lines -- i.e., all objects between
+ (512 bytes, 1024 bytes].
+
+ (bmalloc::Heap::scavenge):
+ (bmalloc::Heap::scavengeSmallPages):
+ (bmalloc::Heap::scavengeLargeObjects): Medium is gone.
+
+ (bmalloc::Heap::allocateSmallBumpRanges): Allow for lines that allocate
+ zero objects. This happens when an object spans more than two lines --
+ the middle lines allocate zero objects.
+
+ Also set the "has free lines" bit to false if we consume the last free
+ line. This needs to be a bit now because not all pages agree on their
+ maximum refcount anymore, so we need an explicit signal for the transition
+ from maximum to maximum - 1.
+
+ (bmalloc::Heap::allocateSmallPage): This code didn't change; I just removed
+ the medium code.
+
+ (bmalloc::Heap::deallocateSmallLine): Changed the algorithm to check
+ hasFreeLines. See allocateSmallBumpRanges.
+
+ (bmalloc::Heap::scavengeMediumPages): Deleted.
+ (bmalloc::Heap::allocateMediumBumpRanges): Deleted.
+ (bmalloc::Heap::allocateMediumPage): Deleted.
+ (bmalloc::Heap::deallocateMediumLine): Deleted.
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::derefMediumLine): Deleted.
+
+ * bmalloc/LargeChunk.h:
+ (bmalloc::LargeChunk::get):
+ (bmalloc::LargeChunk::endTag):
+ * bmalloc/Line.h: No more medium.
+
+ * bmalloc/MediumChunk.h: Removed.
+ * bmalloc/MediumLine.h: Removed.
+ * bmalloc/MediumPage.h: Removed.
+ * bmalloc/MediumTraits.h: Removed.
+
+ * bmalloc/ObjectType.cpp:
+ (bmalloc::objectType):
+ * bmalloc/ObjectType.h:
+ (bmalloc::isSmall):
+ (bmalloc::isXLarge):
+ (bmalloc::isSmallOrMedium): Deleted.
+ (bmalloc::isMedium): Deleted. No more medium.
+
+ * bmalloc/Page.h:
+ (bmalloc::Page::sizeClass):
+ (bmalloc::Page::setSizeClass):
+ (bmalloc::Page::hasFreeLines):
+ (bmalloc::Page::setHasFreeLines): Add the free lines bit. You get better
+ codegen if you make it the low bit, since ref / deref can then add / sub
+ 2. So do that.
+
+ * bmalloc/Sizes.h:
+ (bmalloc::Sizes::sizeClass): Expand the small size class to include the
+ medium size class.
+
+ * bmalloc/SuperChunk.h:
+ (bmalloc::SuperChunk::SuperChunk):
+ (bmalloc::SuperChunk::smallChunk):
+ (bmalloc::SuperChunk::largeChunk):
+ (bmalloc::SuperChunk::mediumChunk): Deleted. No more medium.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::grow):
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateSmallPage): Set the has free lines bit before
+ returning a Page to the Heap since this is the correct default state
+ when we first allocate a page.
+
+ (bmalloc::VMHeap::allocateMediumPage): Deleted.
+ (bmalloc::VMHeap::deallocateMediumPage): Deleted.
+
+2016-02-19 Michael Saboff <msaboff@apple.com>
+
+ bmalloc: Unify VMHeap and Heap LargeObjects free lists to reduce fragmentation
+ https://bugs.webkit.org/show_bug.cgi?id=154192
+
+ Reviewed by Geoffrey Garen.
+
+ Change the operation of Heap and VMHeap LargeObject free lists.
+ Renamed Owner to VMState to track the state of each LargeObject.
+ Physical - The pages have been allocated.
+ Virtual - The pages have not been allocated.
+ Mixed - The object contains a mixture of Physical and Virtual pages.
+ VMState uses one bit each for Physical and Virtual to simplify merging states
+ when merging two adjacent blocks. This change enforces the rule that objects in
+ the Heap free list must have have the Physical bit set in their VMState while objects
+ in the VMHeap free list must have the Physical bit clear. Thie means that the Heap
+ can have LargeObjects in Physical or Mixed VMState, but the VMHeap's free list can
+ only contain Virtual LargeObjects.
+
+ In both Heap::allocateLarge(), we now allocate physical pages if the LargeObject we
+ pull from the free list has any Virtual pages before we possilby split the
+ object. When we merge objects, the result might be made up of Mixed page allocations.
+ When allocating a Mixed LargeObject, we need to allocate memory for them as well.
+ The scavenger deallocates both Physical and Mixed LargeObjects, placing them back into
+ the VMHeap's free list.
+
+ When we allocate or deallocate Mixed LargeObjects, there are pages that within these
+ objects that will be redundantly modified. It would require additional metadata to
+ eliminate this redundancy.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/BoundaryTag.h:
+ (bmalloc::BoundaryTag::vmState): New helper.
+ (bmalloc::BoundaryTag::setVMState): New helper.
+ (bmalloc::BoundaryTag::owner): Deleted.
+ (bmalloc::BoundaryTag::setOwner): Deleted.
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::splitAndAllocate): New helpers.
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::vmState): New helper.
+ (bmalloc::LargeObject::setVMState): New helper.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::splitAndAllocate): New helpers.
+ (bmalloc::Heap::allocateLarge):
+ (bmalloc::Heap::deallocatePhysicalPages): Refactored from VMHeap::deallocateLargeObjectMemory.
+
+ * bmalloc/FreeList.cpp:
+ (bmalloc::FreeList::takeGreedy):
+ (bmalloc::FreeList::take):
+ (bmalloc::FreeList::removeInvalidAndDuplicateEntries):
+ * bmalloc/FreeList.h:
+ (bmalloc::FreeList::FreeList):
+ (bmalloc::FreeList::push):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::scavengeLargeObjects):
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::isValidAndFree):
+ (bmalloc::LargeObject::validateSelf):
+ * bmalloc/SegregatedFreeList.cpp:
+ (bmalloc::SegregatedFreeList::SegregatedFreeList): Changed to initialize our required Physical state.
+ * bmalloc/SegregatedFreeList.h:
+ (bmalloc::SegregatedFreeList::SegregatedFreeList):
+ (bmalloc::SegregatedFreeList::insert):
+ (bmalloc::SegregatedFreeList::takeGreedy):
+ (bmalloc::SegregatedFreeList::take):
+ Replaced Owner parameters and checks with VMState::HasPhysical.
+
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::prevCanMerge): Removed owner from tests.
+ (bmalloc::LargeObject::nextCanMerge): Removed owner from tests.
+ (bmalloc::LargeObject::merge): Removed owner from tests. Updated to merge VMStates andset the
+ VMState after the merge.
+
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::owner): Deleted.
+ (bmalloc::LargeObject::setOwner): Deleted.
+
+ * bmalloc/Owner.h: Removed.
+
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmAllocatePhysicalPagesSloppy): Changed to round begin down to eliminate the left to right
+ allocation constraint.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::grow): Large space managed like small or medium as a vector of LargeChunks.
+ (bmalloc::VMHeap::VMHeap): Changed to initialize our required Physical state.
+
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateLargeObject): These no longer allocate memory.
+ (bmalloc::VMHeap::deallocateLargeObject): Removed setOwner. Now we set the VMState after any merges.
+
+ * bmalloc/VMState.h: Copied from Source/bmalloc/bmalloc/Owner.h.
+ (bmalloc::VMState::VMState):
+ (bmalloc::VMState::hasPhysical):
+ (bmalloc::VMState::hasVirtual):
+ (bmalloc::VMState::merge):
+ (bmalloc::VMState::operator ==):
+ (bmalloc::VMState::operator unsigned):
+ New class with various helpers.
+
+2016-02-12 Michael Saboff <msaboff@apple.com>
+
+ BASSERTs added in r196421 are causing debug test failures
+ https://bugs.webkit.org/show_bug.cgi?id=154113
+
+ Reviewed by Geoffrey Garen.
+
+ In VMHeap::deallocateLargeObject(), we drop the lock to deallocate the physical pages.
+ If the scavenger thread is running at the same time a synchronous call to scavenge()
+ comes in, we could call VMHeap::deallocateLargeObject() for an adjacent object while the
+ lock in the other thread is dropped. We fix this by checking for adjacent objects we
+ can merge with and loop if we have one.
+
+ * bmalloc/FreeList.h:
+ (bmalloc::FreeList::push): Added BASSERT to catch adding unmerged free objects
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateLarge): Changed to use nextCanMerge().
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::prevCanMerge): Repurposed prevIsAllocated.
+ (bmalloc::LargeObject::nextCanMerge): Repurposed nextIsAllocated.
+ (bmalloc::LargeObject::prevIsAllocated): Deleted.
+ (bmalloc::LargeObject::nextIsAllocated): Deleted.
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateLargeObject): Moved adding the extra object back to the free list
+ to after we set the object we'll return as being allocated.
+ (bmalloc::VMHeap::deallocateLargeObject):
+
+2016-02-12 Mark Lam <mark.lam@apple.com>
+
+ Make BCRASH() use breakpoint traps too for non-debug OS(DARWIN).
+ https://bugs.webkit.org/show_bug.cgi?id=154184
+
+ Reviewed by Saam Barati.
+
+ This makes it behave consistently with WTFCrash().
+
+ * bmalloc/BAssert.h:
+ * bmalloc/BPlatform.h:
+
+2016-02-11 Michael Saboff <msaboff@apple.com>
+
+ Unreviewed build fix after r196421.
+
+ Removed BASSERTs that are firing to eliminate Debug build crashes. I'll debug locally and
+ enable or alter after the issue is understood.
+
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::merge): Removed BASSERTs that are firing.
+
+2016-02-11 Michael Saboff <msaboff@apple.com>
+
+ bmalloc: large aligned allocations will put 1 or 2 free object on free list without merging with free neighbors
+ https://bugs.webkit.org/show_bug.cgi?id=154091
+
+ Reviewed by Geoffrey Garen.
+
+ If we split off any unused free object in the aligned version of Heap::allocateLarge(), we merge them with
+ free neighbors before putting them back on the free list. Added helpers to verify that when we
+ add LargeObjects to the free list their neighbors are allocated.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateLarge): Deleted private helper version and rolled it into the two the
+ two public versions of allocateLarge().
+ * bmalloc/Heap.h:
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::prevIsAllocated): New helper.
+ (bmalloc::LargeObject::nextIsAllocated): New helper.
+ (bmalloc::LargeObject::merge): Check that the merge object has allocated neighbors.
+
+2016-02-05 Saam barati <sbarati@apple.com>
+
+ bmalloc: largeMax calculation is wrong on iOS
+ https://bugs.webkit.org/show_bug.cgi?id=153923
+
+ Reviewed by Mark Lam.
+
+ Our number for largeMax was larger than what we had
+ space to actually allocate inside the LargeChunk. This made
+ it so that we would allocate a large object for something
+ that really should be extra large. Previously:
+ largeMax + sizeof(LargeChunk) > 1MB
+ which meant that when we would grow() to accommodate an allocation
+ of a particular size inside a LargeObject despite the fact that
+ the allocation size would be too large to actually fit in the LargeObject.
+ This would manifest when we had an allocation size in the range:
+ 1MB - sizeof(LargeChunk) < allocation size < largeMax
+
+ We fix this bug by being precise in our calculation of largeMax
+ instead of just assuming largeChunkSize * 99/100 is enough
+ space for the metadata.
+
+ * bmalloc/LargeChunk.h:
+ (bmalloc::LargeChunk::get):
+ * bmalloc/Sizes.h:
+
+2016-01-31 Dan Bernstein <mitz@apple.com>
+
+ [Cocoa] Remove unused definition of HAVE_HEADER_DETECTION_H
+ https://bugs.webkit.org/show_bug.cgi?id=153729
+
+ Reviewed by Sam Weinig.
+
+ After r141700, HAVE_HEADER_DETECTION_H is no longer used.
+
+ * Configurations/Base.xcconfig:
+
+2015-12-19 Dan Bernstein <mitz@apple.com>
+
+ [Mac] WebKit contains dead source code for OS X Mavericks and earlier
+ https://bugs.webkit.org/show_bug.cgi?id=152462
+
+ Reviewed by Alexey Proskuryakov.
+
+ * Configurations/DebugRelease.xcconfig: Removed definition of MACOSX_DEPLOYMENT_TARGET for
+ OS X 10.9.
+
+2015-12-03 Anders Carlsson <andersca@apple.com>
+
+ Remove Objective-C GC support
+ https://bugs.webkit.org/show_bug.cgi?id=151819
+ rdar://problem/23746991
+
+ Reviewed by Dan Bernstein.
+
+ * Configurations/Base.xcconfig:
+
+2015-12-03 Michael Saboff <msaboff@apple.com>
+
+ bmalloc: extra large allocations could be more efficient
+ https://bugs.webkit.org/show_bug.cgi?id=151817
+
+ Reviewed by Geoffrey Garen.
+
+ Reduced the super chunk size from 4MB to 2MB.
+
+ Added path to reallocate() of an extra large object to see if we can extend the allocation.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate):
+ * bmalloc/SegregatedFreeList.h:
+ * bmalloc/Sizes.h:
+ * bmalloc/VMAllocate.h:
+ (bmalloc::tryVMAllocate):
+ (bmalloc::tryVMExtend):
+ (bmalloc::vmAllocate):
+
+2015-11-11 Akos Kiss <akiss@inf.u-szeged.hu>
+
+ bmalloc: Add libdl dependency
+ https://bugs.webkit.org/show_bug.cgi?id=151140
+
+ Reviewed by Csaba Osztrogonác.
+
+ Make sure that the linker links libdl and finds the references to
+ dlopen, dlsym and dlclose in Environment.cpp.
+
+ * CMakeLists.txt:
+
+2015-11-02 Andy Estes <aestes@apple.com>
+
+ [Cocoa] Add tvOS and watchOS to SUPPORTED_PLATFORMS
+ https://bugs.webkit.org/show_bug.cgi?id=150819
+
+ Reviewed by Dan Bernstein.
+
+ This tells Xcode to include these platforms in its Devices dropdown, making it possible to build in the IDE.
+
+ * Configurations/Base.xcconfig:
+
+2015-11-01 Philip Chimento <philip.chimento@gmail.com>
+
+ [GTK] Fix combinations of PLATFORM(GTK) and OS(DARWIN)
+ https://bugs.webkit.org/show_bug.cgi?id=144560
+
+ Reviewed by Darin Adler.
+
+ * PlatformGTK.cmake: Added. This adds Zone.cpp to the PlatformGTK
+ build, on Darwin only. Since there was previously nothing for the
+ build system to do that was specific to the GTK platform in
+ bmalloc, we need to create this file.
+
+2015-10-29 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: AsyncTask should handle destruction
+ https://bugs.webkit.org/show_bug.cgi?id=150648
+
+ Reviewed by Mark Lam.
+
+ So we can use it in more places.
+
+ * bmalloc/AsyncTask.h: Use std::thread instead of pthread because it
+ should be more portable.
+
+ (bmalloc::Function>::AsyncTask): Renamed Signaled to RunRequested for
+ clarity. Added an ExitRequested state.
+
+ (bmalloc::Function>::~AsyncTask): Wait for our child thread to exit
+ before destroying ourselves because our child thread will modify our
+ data (and might modify our client's data). Note that we only need to
+ wait for the last child thread since any prior child thread, having
+ reached the Exited condition, is guaranteed not to read or write any
+ data.
+
+ (bmalloc::Function>::run):
+ (bmalloc::Function>::runSlowCase): Updated for interface changes. Also
+ changed to use our WebKit style for condition signal: Hold the lock
+ during the signal and always notify all. Technically, neither is necessary,
+ but it is easier to understand the code this way, and harder to make
+ mistakes.
+
+ (bmalloc::Function>::threadEntryPoint):
+ (bmalloc::Function>::threadRunLoop): Handle the new ExitRequested state.
+ Technically, this state has no meaningful difference from the Exited
+ state, but it is nice to be explicit.
+
+ (bmalloc::Function>::join): Deleted.
+ (bmalloc::Function>::pthreadEntryPoint): Deleted.
+ (bmalloc::Function>::entryPoint): Deleted.
+
+2015-10-15 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: per-thread cache data structure should be smaller
+ https://bugs.webkit.org/show_bug.cgi?id=150218
+
+ Reviewed by Andreas Kling.
+
+ Reduce the number of entries in the range cache because it's really
+ big, and the bigness only helps in cases of serious fragmentation, and
+ it only saves us a little bit of lock acquisition time.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::scavenge):
+ (bmalloc::Allocator::refillAllocatorSlowCase):
+ (bmalloc::Allocator::refillAllocator):
+ (bmalloc::Allocator::allocateLarge):
+ (bmalloc::Allocator::allocateSlowCase):
+ (bmalloc::Allocator::allocateBumpRangeSlowCase): Deleted.
+ (bmalloc::Allocator::allocateBumpRange): Deleted.
+ * bmalloc/Allocator.h: Pass through the empty allocator and the range
+ cache when refilling, and refill both. Otherwise, we always immediately
+ pop the last item in the range cache, wasting that slot of capacity.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateSmallBumpRanges):
+ (bmalloc::Heap::allocateMediumBumpRanges): Account for the fact that
+ the range cache is no longer big enough to guarantee that it can hold
+ all the ranges in a page.
+
+ (bmalloc::Heap::refillSmallBumpRangeCache): Deleted.
+ (bmalloc::Heap::refillMediumBumpRangeCache): Deleted.
+
+ * bmalloc/Heap.h: Move VMHeap to the end of the object because it
+ contains a lot of unused / wasted space, and we want to pack our data
+ together in memory.
+
+ * bmalloc/Sizes.h: Make the range cache smaller.
+
+2015-10-13 Chris Dumez <cdumez@apple.com>
+
+ Avoid useless copies in range-loops that are using 'auto'
+ https://bugs.webkit.org/show_bug.cgi?id=150091
+
+ Reviewed by Sam Weinig.
+
+ Avoid useless copies in range-loops that are using 'auto'. Also use
+ 'auto*' instead of 'auto' when range values are pointers for clarity.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::processObjectLog):
+
+2015-10-12 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Don't ASSERT that all syscalls succeed
+ https://bugs.webkit.org/show_bug.cgi?id=150047
+ <rdar://problem/22649531>
+
+ Reviewed by Mark Lam.
+
+ madvise can fail due to VM behaviors outside of our control:
+ copy-on-write, fork, mprotect, and other stuff.
+
+ Older darwin kernels sometimes return this error value, and new kernels
+ might again in future.
+
+ We haven't gained much from this ASSERT so far, so let's remove it.
+
+ Perhaps in future we can come up with a scheme that makes madvise
+ never fail, or that responds to failure.
+
+ * bmalloc/Syscall.h:
+
+2015-10-10 Dan Bernstein <mitz@apple.com>
+
+ [iOS] Remove project support for iOS 8
+ https://bugs.webkit.org/show_bug.cgi?id=149993
+
+ Reviewed by Alexey Proskuryakov.
+
+ * Configurations/Base.xcconfig:
+ * Configurations/bmalloc.xcconfig:
+ * Configurations/mbmalloc.xcconfig:
+
+2015-08-31 Michael Catanzaro <mcatanzaro@igalia.com>
+
+ Implement bmalloc::isASanEnabled for generic Unix
+ https://bugs.webkit.org/show_bug.cgi?id=148623
+
+ Reviewed by Geoffrey Garen.
+
+ * bmalloc/BPlatform.h: Add BOS_UNIX to detect whether the OS is a Unix.
+ * bmalloc/Environment.cpp:
+ (bmalloc::isASanEnabled): Implement a runtime check that should work on any Unix.
+
+2015-08-19 Geoffrey Garen <ggaren@apple.com>
+
+ Crash @ bmalloc::Environment::computeIsBmallocEnabled
+ https://bugs.webkit.org/show_bug.cgi?id=148183
+
+ Reviewed by NOBODY Michael Saboff.
+
+ CrashTracer says we have some crashes beneath computeIsBmallocEnabled
+ dereferencing null in strstr. We null check getenv but not
+ _dyld_get_image_name, so deduction indicates that _dyld_get_image_name
+ must be returning null. _dyld_get_image_name isn't really documented,
+ so let's assume it can return null.
+
+ * bmalloc/Environment.cpp:
+ (bmalloc::isASanEnabled): Check _dyld_get_image_name's return value for
+ null because we can't prove it won't be null.
+
+2015-07-24 Geoffrey Garen <ggaren@apple.com>
+
+ vmmap crash at JavaScriptCore: 0x31cd12f6 (the JavaScript malloc zone enumerator)
+ https://bugs.webkit.org/show_bug.cgi?id=147274
+
+ Reviewed by Anders Carlsson.
+
+ It's not really clear why vmmap sometimes fails to read the target
+ process, but we can avoid a crash when it does. This is useful because
+ you'll still get all the non-bmalloc data out of the target process,
+ and bmalloc might not even be relevant to your investigation.
+
+ * bmalloc/Zone.cpp:
+ (bmalloc::remoteRead): Check for failure.
+
+2015-07-24 Geoffrey Garen <ggaren@apple.com>
+
+ JavaScriptCore bmalloc should not register its malloc zone more than once
+ https://bugs.webkit.org/show_bug.cgi?id=147273
+
+ Reviewed by Andreas Kling.
+
+ This was a goof: The Zone constructor, by virtue of running automatically,
+ was registering a Zone inside the analysis process.
+
+ * bmalloc/Zone.cpp:
+ (bmalloc::remoteRead): Clarify that the pointer is remote.
+
+ (bmalloc::enumerator):
+ (bmalloc::Zone::Zone):
+ * bmalloc/Zone.h: Separate the normal constructor and the remote constructor.
+ The remote constructor skips zone registration since its goal is not
+ to register a zone in the current process or do any allocation but rather
+ to mirror the bytes of the zone from the target process.
+
+2015-07-23 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Shrink the super chunk size (again)
+ https://bugs.webkit.org/show_bug.cgi?id=147240
+
+ Reviewed by Andreas Kling.
+
+ Shrinking to 8MB reduced VM exhaustion crashes but did not eliminate them.
+ Let's try 4MB.
+
+ (My previous comment was that the maximum fast object was 2MB. But it
+ was 4MB! Now it's 2MB for realsies.)
+
+ * bmalloc/Sizes.h:
+
+2015-07-03 Dan Bernstein <mitz@apple.com>
+
+ [Xcode] Update some build settings as recommended by Xcode 7
+ https://bugs.webkit.org/show_bug.cgi?id=146597
+
+ Reviewed by Sam Weinig.
+
+ * Configurations/Base.xcconfig: Enabled CLANG_WARN_UNREACHABLE_CODE, GCC_NO_COMMON_BLOCKS,
+ and ENABLE_STRICT_OBJC_MSGSEND. Removed GCC_MODEL_TUNING.
+
+ * bmalloc.xcodeproj/project.pbxproj: Updated LastUpgradeCheck.
+
+2015-07-02 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Shrink the super chunk size
+ https://bugs.webkit.org/show_bug.cgi?id=146519
+
+ Reviewed by Andreas Kling.
+
+ We have lots of reports of crashing due to failed VM allocation on iOS.
+ (This VM limit on iOS is usually 1GB-2GB, and has been as low as 256MB.)
+
+ Shrink the super chunk size in case fragmentation is the reason for
+ VM allocation failure.
+
+ This has the downside that >= 2MB allocations will now be super slow,
+ but they are also super rare (as in never on most websites), so this
+ is probably an OK tradeoff.
+
+ * bmalloc/Sizes.h:
+
+2015-07-01 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: realloc of an XLarge range can unmap adjacent VM ranges
+ https://bugs.webkit.org/show_bug.cgi?id=146535
+
+ Reviewed by Anders Carlsson.
+
+ This bug causes a crash when running fast/css/large-list-of-rules-crash.html
+ with the fix applied for https://bugs.webkit.org/show_bug.cgi?id=146519.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate): Start at object + newSize since starting
+ at object + oldSize means deleting the adjacent VM range.
+
+2015-05-26 Geoffrey Garen <ggaren@apple.com>
+
+ Integer overflow in XLarge allocation (due to unchecked roundUpToMultipleOf)
+ https://bugs.webkit.org/show_bug.cgi?id=145385
+
+ Reviewed by Andreas Kling.
+
+ Added some checking to verify that round-up operations will not overflow
+ a size_t.
+
+ The simplest way to do this was to introduce a notion of xLargeMax, like
+ we have for smallMax, mediumMax, and largeMax. It's a bit surprising at
+ first to think that there is an xLargeMax, since xLarge is what we use
+ to handle the biggest things. But computers have limits, so it makes sense.
+
+ FWIW, TCMalloc used to have an xLargeMax too, which it called kMaxValidPages.
+
+ No test because this bug was found by code inspection and I don't know
+ of a practical way to convince WebKit to make an allocation this large.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::tryAllocate):
+ (bmalloc::Allocator::allocate):
+ (bmalloc::Allocator::reallocate):
+ (bmalloc::Allocator::allocateSlowCase): Check against xLargeMax to avoid
+ overflow when rounding up.
+
+ * bmalloc/BAssert.h: Added support for explicit crashing.
+
+ * bmalloc/Sizes.h:
+
+2015-05-26 Dan Bernstein <mitz@apple.com>
+
+ <rdar://problem/21104551> Update build settings
+
+ Reviewed by Anders Carlsson.
+
+ * Configurations/DebugRelease.xcconfig:
+
+2015-05-23 Dan Bernstein <mitz@apple.com>
+
+ Remove unused definitions of WEBKIT_VERSION_MIN_REQUIRED
+ https://bugs.webkit.org/show_bug.cgi?id=145345
+
+ Reviewed by Sam Weinig.
+
+ * Configurations/Base.xcconfig: Also changed to use $(inherited).
+
+2015-05-07 Geoffrey Garen <ggaren@apple.com>
+
+ Release assert in com.apple.WebKit.WebContent under JavaScriptCore: JSC::JSONProtoFuncStringify
+ https://bugs.webkit.org/show_bug.cgi?id=144758
+
+ Reviewed by Andreas Kling.
+
+ This was an out-of-memory error when trying to shrink a string builder.
+ bmalloc was missing the optimization that allowed realloc() to shrink
+ without copying. So, let's add it.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate): Added Large and XLarge cases for
+ shrinking without copying. This isn't possible for small and medium
+ objects, and probably not very profitable, either.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::findXLarge):
+ (bmalloc::Heap::deallocateXLarge):
+ * bmalloc/Heap.h: Refactored this code to return a reference to an
+ XLarge range. This makes the code reusable, and also makes it easier
+ for realloc() to update metadata.
+
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::split): Allow allocated objects to split because
+ that's what realloc() wants to do, and there's nothing intrinsically
+ wrong with it.
+
+2015-05-07 Dan Bernstein <mitz@apple.com>
+
+ <rdar://problem/19317140> [Xcode] Remove usage of AspenFamily.xcconfig in Source/
+ https://bugs.webkit.org/show_bug.cgi?id=144727
+
+ Reviewed by Darin Adler.
+
+ * Configurations/Base.xcconfig: Dont’s include AspenFamily.xcconfig, and define
+ INSTALL_PATH_PREFIX for the iOS 8.x Simulator.
+
+2015-04-01 Alex Christensen <achristensen@webkit.org>
+
+ Progress towards CMake on Windows and Mac.
+ https://bugs.webkit.org/show_bug.cgi?id=143293
+
+ Reviewed by Filip Pizlo.
+
+ * bmalloc/BAssert.h:
+ Removed ellipses from macros to appease Visual Studio.
+
+2015-03-13 Alex Christensen <achristensen@webkit.org>
+
+ Progress towards CMake on Mac.
+ https://bugs.webkit.org/show_bug.cgi?id=142680
+
+ Reviewed by Gyuyoung Kim.
+
+ * CMakeLists.txt:
+ * PlatformMac.cmake:
+ Added Zone.cpp to Mac CMake builds.
+
+2015-03-12 Geoffrey Garen <ggaren@apple.com>
+
+ Assertion failure in bmalloc::LargeObject::validateSelf on Mavericks Debug layout test bot
+ https://bugs.webkit.org/show_bug.cgi?id=142642
+
+ Reviewed by Michael Saboff.
+
+ The typical backtrace to this crash shows the main thread trying to
+ realloc a large string while a DFG compiler thread tries to
+ free a large vector buffer.
+
+ I believe that this is a race condition -- at least in debug builds --
+ since the main thread will try to validate its object's neighbors
+ without holding a lock, even though those neighbors might be in the
+ midst of changing.
+
+ In general, there may be sneaky times when it is valid to look at an
+ object's metadata without holding the heap lock, but it is best not to
+ do so unless we have a really really good reason to.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate): Take a lock before reading the metadata
+ for this object, since we generally require any access to shared heap
+ metadata to take a lock.
+
+2015-03-10 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: tryFastMalloc shouldn't crash
+ https://bugs.webkit.org/show_bug.cgi?id=142443
+
+ Reviewed by Sam Weinig.
+
+ Rolling back in r181307 with a check for whether bmalloc is enabled, to
+ avoid crashes when running with ASan and GuardMalloc.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::tryAllocate):
+ * bmalloc/Allocator.h:
+ * bmalloc/Cache.cpp:
+ (bmalloc::Cache::tryAllocateSlowCaseNullCache):
+ * bmalloc/Cache.h:
+ (bmalloc::Cache::tryAllocate):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateXLarge):
+ (bmalloc::Heap::tryAllocateXLarge):
+ * bmalloc/Heap.h:
+ * bmalloc/VMAllocate.h:
+ (bmalloc::tryVMAllocate):
+ (bmalloc::vmAllocate):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::tryMalloc):
+ (bmalloc::api::realloc):
+ (bmalloc::api::free):
+
+2015-03-09 Commit Queue <commit-queue@webkit.org>
+
+ Unreviewed, rolling out r181307.
+ https://bugs.webkit.org/show_bug.cgi?id=142525
+
+ Broke ASan tests (Requested by ap on #webkit).
+
+ Reverted changeset:
+
+ "bmalloc: tryFastMalloc shouldn't crash"
+ https://bugs.webkit.org/show_bug.cgi?id=142443
+ http://trac.webkit.org/changeset/181307
+
+2015-03-09 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: tryFastMalloc shouldn't crash
+ https://bugs.webkit.org/show_bug.cgi?id=142443
+
+ Reviewed by Darin Adler.
+
+ Added support for tryMalloc.
+
+ We assume that non-x-large allocations always succeed, and we crash
+ otherwise, since normal allocation failure will just cause the next
+ non-try allocation or internal metadata allocation to fail, and it's
+ hard and not really useful to keep limping along after that. But
+ extra-large allocations can meaningfully fail, and we can recover.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateXLarge):
+ (bmalloc::Heap::tryAllocateXLarge):
+ * bmalloc/Heap.h: Added support for non-crashy x-large allocation.
+
+ * bmalloc/VMAllocate.h:
+ (bmalloc::tryVMAllocate):
+ (bmalloc::vmAllocate): Added support for non-crashy VM allocation.
+
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::tryMalloc):
+ (bmalloc::api::realloc):
+ (bmalloc::api::free): Tried to clarify our behavior with some comments.
+ Unfortunately, calling what we do "malloc" is still not quite right, since
+ malloc returns null on failure and we don't.
+
+2015-03-03 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Don't branch when setting the owner of a large object
+ https://bugs.webkit.org/show_bug.cgi?id=142241
+
+ Reviewed by Andreas Kling.
+
+ * bmalloc/BoundaryTag.h:
+ (bmalloc::BoundaryTag::owner):
+ (bmalloc::BoundaryTag::setOwner):
+
+2015-03-03 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
+ https://bugs.webkit.org/show_bug.cgi?id=141802
+
+ Reviewed by Andreas Kling.
+
+ Re-enabled this feature on iOS, now that the iOS crash should be fixed.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::grow):
+ * bmalloc/VMHeap.h:
+
+2015-03-03 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Added missing features to the malloc zone introspection API
+ https://bugs.webkit.org/show_bug.cgi?id=142235
+
+ Reviewed by Andreas Kling.
+
+ This should fix the crash we saw on the iOS PLT bot
+ (c.f. http://trac.webkit.org/changeset/180604).
+
+ * bmalloc/Zone.cpp:
+ (bmalloc::good_size):
+ (bmalloc::check):
+ (bmalloc::print):
+ (bmalloc::log):
+ (bmalloc::force_lock):
+ (bmalloc::force_unlock):
+ (bmalloc::statistics):
+ (bmalloc::size):
+ (bmalloc::enumerator): Provide all of these functions since they are called
+ indiscriminately on all zones.
+
+ (bmalloc::Zone::Zone):
+ (bmalloc::Zone::size): Deleted.
+ (bmalloc::Zone::enumerator): Deleted. Moved these functions out of the
+ Zone class since they can stand alone.
+
+ * bmalloc/Zone.h:
+
+2015-03-03 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
+ https://bugs.webkit.org/show_bug.cgi?id=141802
+
+ Reviewed by Andreas Kling.
+
+ Rolling back in but disabled on iOS until I can debug why the iOS PLT crashes.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::grow):
+ * bmalloc/VMHeap.h:
+ * bmalloc/Zone.cpp:
+ (bmalloc::Zone::size):
+ (bmalloc::Zone::Zone):
+ * bmalloc/Zone.h:
+
+2015-03-03 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Miscellaneous cleanup
+ https://bugs.webkit.org/show_bug.cgi?id=142231
+
+ Reviewed by Andreas Kling.
+
+ No performance change -- maybe a tiny reduction in memory use.
+
+ * bmalloc/Heap.cpp: Moved the sleep function into StaticMutex, since
+ it's a helper for working with mutexes.
+
+ (bmalloc::Heap::scavenge): Make sure to wait before we start any
+ scavenging, since individual scavenging functions now always scavenge
+ at least one page before waiting themselves.
+
+ (bmalloc::Heap::scavengeSmallPages):
+ (bmalloc::Heap::scavengeMediumPages):
+ (bmalloc::Heap::scavengeLargeObjects): Use the new wait helper to
+ simplify this code. Also, we now require our caller to wait until at
+ least one deallocation is desirable. This simplifies our loop.
+
+ (bmalloc::Heap::allocateSmallPage):
+ (bmalloc::Heap::allocateMediumPage):
+ (bmalloc::Heap::allocateXLarge):
+ (bmalloc::Heap::allocateLarge): Don't freak out any time the heap does
+ an allocation. Only consider the heap to be growing if it actually needs
+ to allocate new VM. This allows us to shrink the heap back down from a
+ high water mark more reliably even if heap activity continues.
+
+ (bmalloc::sleep): Deleted.
+ (bmalloc::Heap::scavengeLargeRanges): Renamed to match our use of
+ "LargeObject".
+
+ * bmalloc/Heap.h:
+
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::operator bool): Added to simplify a while loop.
+
+ * bmalloc/StaticMutex.h:
+ (bmalloc::sleep):
+ (bmalloc::waitUntilFalse): New helper for waiting until a condition
+ becomes reliably false.
+
+ * bmalloc/Vector.h:
+ (bmalloc::Vector<T>::~Vector): Oops! Don't deallocate the null pointer.
+ We don't actually run any Vector destructors, but an iteration of this
+ patch did, and then crashed. So, let's fix that.
+
+2015-03-02 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Eagerly remove allocated objects from the free list
+ https://bugs.webkit.org/show_bug.cgi?id=142194
+
+ Reviewed by Andreas Kling.
+
+ This reduces the pressure to garbage collect the free list.
+
+ Might be a 1% speedup on MallocBench.
+
+ * bmalloc/FreeList.cpp: Put this comment at the top of the file instead
+ of repeating it inside of each function. Tried to clarify the details.
+
+ (bmalloc::FreeList::takeGreedy): Matched the other iteration code in this
+ file for consistency -- even though either direction works fine in this
+ function.
+
+ (bmalloc::FreeList::take): Change to iterate from low to high so that we
+ can maintain an index into the vector that is not disturbed even if we
+ pop from the middle (which invalidates the last index in the vector).
+
+ Decrement i when popping from the middle to make sure that we don't
+ skip the next item after popping.
+
+ (bmalloc::FreeList::removeInvalidAndDuplicateEntries): Ditto.
+
+2015-02-27 Ryosuke Niwa <rniwa@webkit.org>
+
+ Fixed a typo in the previous commit.
+
+ * bmalloc/BoundaryTag.h:
+ (bmalloc::BoundaryTag::setOwner):
+
+2015-02-27 Ryosuke Niwa <rniwa@webkit.org>
+
+ EFL build fix after r180797.
+
+ * bmalloc/BoundaryTag.h:
+ (bmalloc::BoundaryTag::owner):
+ (bmalloc::BoundaryTag::setOwner):
+
+2015-02-27 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Pathological madvise churn on the free(malloc(x)) benchmark
+ https://bugs.webkit.org/show_bug.cgi?id=142058
+
+ Reviewed by Andreas Kling.
+
+ The churn was caused by repeatedly splitting an object with physical
+ pages from an object without, and then merging them back together again.
+ The merge would conservatively forget that we had physical pages, forcing
+ a new call to madvise on the next allocation.
+
+ This patch more strictly segregates objects in the heap from objects in
+ the VM heap, with these changes:
+
+ (1) Objects in the heap are not allowed to merge with objects in the VM
+ heap, and vice versa -- since that would erase our precise knowledge of
+ which physical pages had been allocated.
+
+ (2) The VM heap is exclusively responsible for allocating and deallocating
+ physical pages.
+
+ (3) The heap free list must consider entries for objects that are in the
+ VM heap to be invalid, and vice versa. (This condition can arise
+ because the free list does not eagerly remove items.)
+
+ With these changes, we can know that any valid object in the heap's free
+ list already has physical pages, and does not need to call madvise.
+
+ Note that the VM heap -- as before -- might sometimes contain ranges
+ or pieces of ranges that have physical pages, since we allow splitting
+ of ranges at granularities smaller than the VM page size. These ranges
+ can eventually merge with ranges in the heap during scavenging.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+
+ * bmalloc/BoundaryTag.h:
+ (bmalloc::BoundaryTag::owner):
+ (bmalloc::BoundaryTag::setOwner):
+ (bmalloc::BoundaryTag::initSentinel):
+ (bmalloc::BoundaryTag::hasPhysicalPages): Deleted.
+ (bmalloc::BoundaryTag::setHasPhysicalPages): Deleted. Replaced the concept
+ of "has physical pages" with a bit indicating which heap owns the large
+ object. This is a more precise concept, since the old bit was really a
+ Yes / Maybe bit.
+
+ * bmalloc/Deallocator.cpp:
+
+ * bmalloc/FreeList.cpp: Adopt
+ (bmalloc::FreeList::takeGreedy):
+ (bmalloc::FreeList::take):
+ (bmalloc::FreeList::removeInvalidAndDuplicateEntries):
+ * bmalloc/FreeList.h:
+ (bmalloc::FreeList::push): Added API for considering the owner when
+ deciding if a free list entry is valid.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap): Adopt new API.
+
+ (bmalloc::Heap::scavengeLargeRanges): Scavenge all ranges with no minimum,
+ since some ranges might be able to merge with ranges in the VM heap, and
+ they won't be allowed to until we scavenge them.
+
+ (bmalloc::Heap::allocateSmallPage):
+ (bmalloc::Heap::allocateMediumPage):
+ (bmalloc::Heap::allocateLarge): New VM heap API makes this function
+ simpler, since we always get back physical pages now.
+
+ * bmalloc/Heap.h:
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::end):
+ (bmalloc::LargeObject::owner):
+ (bmalloc::LargeObject::setOwner):
+ (bmalloc::LargeObject::isValidAndFree):
+ (bmalloc::LargeObject::merge): Do not merge objects across heaps since
+ that causes madvise churn.
+ (bmalloc::LargeObject::validateSelf):
+ (bmalloc::LargeObject::init):
+ (bmalloc::LargeObject::hasPhysicalPages): Deleted.
+ (bmalloc::LargeObject::setHasPhysicalPages): Deleted. Propogate the Owner API.
+
+ * bmalloc/Owner.h: Added.
+
+ * bmalloc/SegregatedFreeList.cpp:
+ (bmalloc::SegregatedFreeList::SegregatedFreeList):
+ (bmalloc::SegregatedFreeList::insert):
+ (bmalloc::SegregatedFreeList::takeGreedy):
+ (bmalloc::SegregatedFreeList::take):
+ * bmalloc/SegregatedFreeList.h: Propogate the owner API.
+
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmDeallocatePhysicalPagesSloppy):
+ (bmalloc::vmAllocatePhysicalPagesSloppy): Clarified these functions and
+ removed an edge case.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::VMHeap):
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateSmallPage):
+ (bmalloc::VMHeap::allocateMediumPage):
+ (bmalloc::VMHeap::allocateLargeObject):
+ (bmalloc::VMHeap::deallocateLargeObject): Be sure to give each object
+ a new chance to merge, since it might have been prohibited from merging
+ before by virtue of not being in the VM heap.
+
+ (bmalloc::VMHeap::allocateLargeRange): Deleted.
+ (bmalloc::VMHeap::deallocateLargeRange): Deleted.
+
+2015-02-26 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Large object free list can grow infinitely
+ https://bugs.webkit.org/show_bug.cgi?id=142055
+
+ Reviewed by Andreas Kling.
+
+ By design, we don't eagerly remove large objects from the free list.
+ This creates two simple pathologies:
+
+ (1) If you free and then allocate the same object repeatedly, it will
+ duplicate itself in the free list repeatedly. Since it is never
+ invalid at the time of allocation, it will never be removed.
+
+ (2) If you split and then merge the same object repeatedly, it will
+ duplicate its split sibling in the free list repeatedly. If its
+ sibling is in a separate free list size class, it will never be
+ consulted at the time of allocation, so it will never be removed.
+
+ So, a simple "while (1) { free(malloc(x)); }" causes infinite memory
+ use in the free list.
+
+ The solution in this patch is a simple helper to remove garbage from the
+ free list if it grows too large. This pathology is not common, so the
+ cost is OK.
+
+ Long-term, perhaps we should rethink the laziness of these free lists.
+
+ * bmalloc/BoundaryTag.h:
+ (bmalloc::BoundaryTag::isMarked):
+ (bmalloc::BoundaryTag::setMarked): New bit, used by free list GC.
+
+ * bmalloc/FreeList.cpp:
+ (bmalloc::FreeList::removeInvalidAndDuplicateEntries): The GC algorithm.
+
+ * bmalloc/FreeList.h:
+ (bmalloc::FreeList::FreeList):
+ (bmalloc::FreeList::push): Invoke the GC if we're getting huge.
+
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::isMarked):
+ (bmalloc::LargeObject::setMarked):
+ (bmalloc::LargeObject::validateSelf): Expose the new bit.
+
+ * bmalloc/Sizes.h: New constant to control GC frequency.
+
+2015-02-26 Csaba Osztrogonác <ossy@webkit.org>
+
+ URTBF after r180693.
+
+ * CMakeLists.txt:
+
+2015-02-26 Geoffrey Garen <ggaren@apple.com>
+
+ Try to fix the Mac build.
+
+ Unreviewed.
+
+ * bmalloc.xcodeproj/project.pbxproj: Make FreeList.h available.
+
+2015-02-26 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Refactored SegregatedFreeList and BoundaryTag::init
+ https://bugs.webkit.org/show_bug.cgi?id=142049
+
+ Reviewed by Anders Carlsson.
+
+ Split out a FreeList class from SegregatedFreeList. This will make it
+ easier to add behaviors on free list insertion and removal -- and it's
+ probably how I should have designed things at the start.
+
+ Moved BoundaryTag::init into LargeObject, since all the related logic
+ lives in LargeObject now too, and this allows us to remove BoundaryTagInlines.h.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/BoundaryTagInlines.h: Removed.
+ * bmalloc/FreeList.cpp: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.cpp.
+ (bmalloc::FreeList::takeGreedy):
+ (bmalloc::FreeList::take):
+ (bmalloc::SegregatedFreeList::SegregatedFreeList): Deleted.
+ (bmalloc::SegregatedFreeList::insert): Deleted.
+ (bmalloc::SegregatedFreeList::takeGreedy): Deleted.
+ (bmalloc::SegregatedFreeList::take): Deleted.
+ * bmalloc/FreeList.h: Copied from Source/bmalloc/bmalloc/SegregatedFreeList.h.
+ (bmalloc::FreeList::push):
+ * bmalloc/LargeObject.h:
+ (bmalloc::LargeObject::init):
+ * bmalloc/SegregatedFreeList.cpp:
+ (bmalloc::SegregatedFreeList::SegregatedFreeList):
+ (bmalloc::SegregatedFreeList::insert):
+ (bmalloc::SegregatedFreeList::takeGreedy):
+ (bmalloc::SegregatedFreeList::take):
+ * bmalloc/SegregatedFreeList.h:
+ * bmalloc/Sizes.h:
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::grow):
+
+2015-02-26 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: free up a bit in BoundaryTag
+ https://bugs.webkit.org/show_bug.cgi?id=142048
+
+ Reviewed by Brady Eidson.
+
+ We were wasting a bit by accident, and I need one now.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::rightShift): Deleted. Not needed, now that I've simplified
+ the math.
+
+ * bmalloc/BoundaryTag.h: Since each boundary tag bucket is 1024 bytes
+ long, the maximum offset into a bucket is 1023.
+
+ You need 5 bits to count up to 1024, but only 4 to count up to 1023.
+
+ Math is hard.
+
+ (bmalloc::BoundaryTag::compactBegin): Switched to division because it
+ is simpler, and easier to match up with our ASSERT. The compiler will
+ turn division by constant power of two into a shift for us.
+
+ (bmalloc::BoundaryTag::setRange): Added an ASSERT for compactBegin
+ because we do encode it, so we should ASSERT that encoding did not
+ lose information.
+
+ * bmalloc/Sizes.h: Shifting is no longer used since we use division
+ instead.
+
+2015-02-24 Stephanie Lewis <slewis@apple.com>
+
+ Rolling out http://trac.webkit.org/changeset/180430 as it causes the PLT to crash.
+ <rdar://problem/19948015>
+
+ Unreviewed.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::grow):
+ * bmalloc/VMHeap.h:
+ * bmalloc/Zone.cpp:
+ (bmalloc::Zone::Zone):
+ (bmalloc::Zone::size): Deleted.
+ * bmalloc/Zone.h:
+
+2015-02-24 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Added a little more abstraction for large objects
+ https://bugs.webkit.org/show_bug.cgi?id=141978
+
+ Reviewed by Sam Weinig.
+
+ Previously, each client needed to manage the boundary tags of
+ a large object using free functions. This patch introduces a LargeObject
+ class that does things a little more automatically.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate): Use the new LargeObject class.
+
+ * bmalloc/BeginTag.h:
+ (bmalloc::BeginTag::isInFreeList): Deleted. Moved this logic into the
+ LargeObject class.
+
+ * bmalloc/BoundaryTag.h:
+ (bmalloc::BoundaryTag::isSentinel):
+ (bmalloc::BoundaryTag::compactBegin):
+ (bmalloc::BoundaryTag::setRange):
+ (bmalloc::BoundaryTag::initSentinel): Added an explicit API for sentinels,
+ which we used to create and test for implicitly.
+
+ * bmalloc/BoundaryTagInlines.h:
+ (bmalloc::BoundaryTag::init):
+ (bmalloc::validate): Deleted.
+ (bmalloc::validatePrev): Deleted.
+ (bmalloc::validateNext): Deleted.
+ (bmalloc::BoundaryTag::mergeLeft): Deleted.
+ (bmalloc::BoundaryTag::mergeRight): Deleted.
+ (bmalloc::BoundaryTag::merge): Deleted.
+ (bmalloc::BoundaryTag::deallocate): Deleted.
+ (bmalloc::BoundaryTag::split): Deleted.
+ (bmalloc::BoundaryTag::allocate): Deleted. Moved this logic into the
+ LargeObject class.
+
+ * bmalloc/EndTag.h:
+ (bmalloc::EndTag::init):
+ (bmalloc::EndTag::operator=): Deleted. Re-reading this code, I found
+ special behavior in the assignment operator to be a surprising API.
+ So, I replaced the assignment operation with an explicit initializing
+ function.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::scavengeLargeRanges):
+ (bmalloc::Heap::allocateXLarge):
+ (bmalloc::Heap::findXLarge):
+ (bmalloc::Heap::deallocateXLarge):
+ (bmalloc::Heap::allocateLarge):
+ (bmalloc::Heap::deallocateLarge):
+ * bmalloc/Heap.h: No behavior changes here -- just adopting the
+ LargeObject interface.
+
+ * bmalloc/LargeObject.h: Added.
+ (bmalloc::LargeObject::operator!):
+ (bmalloc::LargeObject::begin):
+ (bmalloc::LargeObject::size):
+ (bmalloc::LargeObject::range):
+ (bmalloc::LargeObject::LargeObject):
+ (bmalloc::LargeObject::setFree):
+ (bmalloc::LargeObject::isFree):
+ (bmalloc::LargeObject::hasPhysicalPages):
+ (bmalloc::LargeObject::setHasPhysicalPages):
+ (bmalloc::LargeObject::isValidAndFree):
+ (bmalloc::LargeObject::merge):
+ (bmalloc::LargeObject::split):
+ (bmalloc::LargeObject::validateSelf):
+ (bmalloc::LargeObject::validate): Moved this code into a class, out of
+ BoundaryTag free functions.
+
+ New to the class are these features:
+
+ (1) Every reference to an object is validated upon creation and use.
+
+ (2) There's an explicit API for "This is a reference to an object
+ that might be stale (the DoNotValidate API)".
+
+ (3) The begin and end tags are kept in sync automatically.
+
+ * bmalloc/SegregatedFreeList.cpp:
+ (bmalloc::SegregatedFreeList::insert):
+ (bmalloc::SegregatedFreeList::takeGreedy):
+ (bmalloc::SegregatedFreeList::take):
+ * bmalloc/SegregatedFreeList.h: Adopt the LargeObject interface.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::grow):
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateLargeRange):
+ (bmalloc::VMHeap::deallocateLargeRange): Adopt the LargeObject interface.
+
+2015-02-20 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
+ https://bugs.webkit.org/show_bug.cgi?id=141802
+
+ Reviewed by Andreas Kling.
+
+ Rolling back in with a fix for a crash seen while using GuardMalloc.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::grow):
+ * bmalloc/VMHeap.h:
+ * bmalloc/Zone.cpp: Re-land the old patch.
+
+ (bmalloc::Zone::size): Be sure to implement the size() function since
+ it's accessible indirectly via the malloc_zone_from_ptr public API --
+ and GuardMalloc calls it all the time.
+
+ (bmalloc::Zone::Zone):
+ * bmalloc/Zone.h: Re-land the old patch.
+
+2015-02-19 Commit Queue <commit-queue@webkit.org>
+
+ Unreviewed, rolling out r180363.
+ https://bugs.webkit.org/show_bug.cgi?id=141814
+
+ Caused >50 crashes when running LayoutTests in GuardMalloc or
+ ASAN modes. (Requested by jernoble on #webkit).
+
+ Reverted changeset:
+
+ "bmalloc should implement malloc introspection (to stop false-
+ positive leaks when MallocStackLogging is off)"
+ https://bugs.webkit.org/show_bug.cgi?id=141802
+ http://trac.webkit.org/changeset/180363
+
+2015-02-19 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
+ https://bugs.webkit.org/show_bug.cgi?id=141802
+
+ Reviewed by Andreas Kling.
+
+ Fixed a last-minute type.
+
+ The macro is OS, not PLATFORM.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::grow):
+ * bmalloc/VMHeap.h:
+ * bmalloc/Zone.h:
+
+2015-02-19 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc should implement malloc introspection (to stop false-positive leaks when MallocStackLogging is off)
+ https://bugs.webkit.org/show_bug.cgi?id=141802
+
+ Reviewed by Andreas Kling.
+
+ This patch does the bare minimum to stop false positive leaks from
+ being reported by the Darwin leaks tool. We register each super chunk
+ as a single object, and then request that the leaks tool scan it.
+
+ * bmalloc.xcodeproj/project.pbxproj: Added an abstraction for the malloc
+ zone introspection API.
+
+ * bmalloc/Algorithm.h: Missing #include.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::grow):
+ * bmalloc/VMHeap.h: Adopt the new abstraction.
+
+ * bmalloc/Zone.cpp: Added.
+ (bmalloc::remoteRead): Helper for reading an object out of another process.
+ (bmalloc::Zone::enumerator):
+ (bmalloc::Zone::Zone): Register a malloc zone so that we will participate
+ in introspection.
+
+ * bmalloc/Zone.h: Added.
+ (bmalloc::Zone::superChunks):
+ (bmalloc::Zone::addSuperChunk): Use a non-dynamically-allocated vector
+ since our dynamic allocations will not be scanned by leaks since they
+ will have the malloc VM tag.
+
+2015-02-18 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: VMHeap should keep a record of all of its VM ranges (for malloc introspection)
+ https://bugs.webkit.org/show_bug.cgi?id=141759
+
+ Reviewed by Andreas Kling.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/SuperChunk.h: Added.
+ (bmalloc::SuperChunk::create):
+ (bmalloc::SuperChunk::SuperChunk):
+ (bmalloc::SuperChunk::smallChunk):
+ (bmalloc::SuperChunk::mediumChunk):
+ (bmalloc::SuperChunk::largeChunk): Factored out super chunk creation
+ into a separate class, for clarity and type safety.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::grow):
+ (bmalloc::VMHeap::allocateSuperChunk): Renamed "allocateSuperChunk" to
+ "grow" because Andreas found "allocateSuperChunk" to be unclear.
+
+ * bmalloc/VMHeap.h: Track all our VM ranges. We will use this information
+ for malloc introspection.
+
+ (bmalloc::VMHeap::allocateSmallPage):
+ (bmalloc::VMHeap::allocateMediumPage):
+ (bmalloc::VMHeap::allocateLargeRange): Updated for renames.
+
+2015-02-18 Zan Dobersek <zdobersek@igalia.com>
+
+ Build bmalloc through CMake as a static library. It's then linked either
+ into the WTF library (if built as a shared library) or into the JSC and
+ WebKit2 libraries. There's no need to build it as a standalone shared library.
+
+ Rubber-stamped by Carlos Garcia Campos.
+
+ * CMakeLists.txt:
+
+2015-02-13 Gyuyoung Kim <gyuyoung.kim@samsung.com>
+
+ [BMalloc] Add a FIXME comment for memory alignas
+ https://bugs.webkit.org/show_bug.cgi?id=141556
+
+ Reviewed by Csaba Osztrogonác.
+
+ * bmalloc/Chunk.h: Add a FIXME comment.
+ * bmalloc/LargeChunk.h: ditto.
+
+2015-02-11 Csaba Osztrogonác <ossy@webkit.org>
+
+ bmalloc buildfix on 32 bit Linux (x86/ARM)
+ https://bugs.webkit.org/show_bug.cgi?id=141472
+
+ Reviewed by Gyuyoung Kim.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::roundUpToMultipleOf):
+ * bmalloc/FixedVector.h:
+ (bmalloc::FixedVector::clear):
+ * bmalloc/Sizes.h:
+ (bmalloc::Sizes::sizeClass):
+
+2015-02-11 Gyuyoung Kim <gyuyoung.kim@samsung.com>
+
+ [EFL][GTK] Use bmalloc instead of tcmalloc
+ https://bugs.webkit.org/show_bug.cgi?id=140162
+
+ Reviewed by Carlos Garcia Campos.
+
+ Support to use bmalloc on EFL and GTK ports.
+
+ * CMakeLists.txt: Added.
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocate):
+ Fix unused return value caused by posix_memalign().
+ * bmalloc/AsyncTask.h:
+ * bmalloc/BoundaryTag.h:
+ (bmalloc::BoundaryTag::clear):
+ * bmalloc/Chunk.h:
+ Change Traits::Page with Traits::PageType in order to fix
+ -fpermitive build error on EFL and GTK port.
+ * bmalloc/EndTag.h:
+ (bmalloc::EndTag::operator=):
+ * bmalloc/Line.h: ditto.
+ * bmalloc/MediumTraits.h:
+ * bmalloc/Page.h: ditto.
+ * bmalloc/PerThread.h:
+ EFL port doesn't support __has_include definition yet.
+ Define HAVE_PTHREAD_MACHDEP_H according to check if __has_include is supported.
+ * bmalloc/SmallTraits.h: ditto.
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmDeallocatePhysicalPages):
+ (bmalloc::vmAllocatePhysicalPages):
+ * bmalloc/Vector.h:
+ (bmalloc::Vector<T>::push):
+ (bmalloc::Vector<T>::reallocateBuffer):
+
+2015-01-31 Sam Weinig <sam@webkit.org>
+
+ Remove even more Mountain Lion support
+ https://bugs.webkit.org/show_bug.cgi?id=141124
+
+ Reviewed by Alexey Proskuryakov.
+
+ * Configurations/Base.xcconfig:
+ * Configurations/DebugRelease.xcconfig:
+
+2015-01-30 Geoffrey Garen <ggaren@apple.com>
+
+ GC marking threads should clear malloc caches
+ https://bugs.webkit.org/show_bug.cgi?id=141097
+
+ Reviewed by Andreas Kling.
+
+ Split the scavenging API into per-thread vs global, so that you can
+ request to scavenge your own thread without scavenging the whole heap.
+
+ * bmalloc/Cache.cpp:
+ (bmalloc::Cache::scavenge):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::scavengeThisThread):
+ (bmalloc::api::scavenge):
+
+2015-01-28 Dana Burkart <dburkart@apple.com>
+
+ Move ASan flag settings from DebugRelease.xcconfig to Base.xcconfig
+ https://bugs.webkit.org/show_bug.cgi?id=136765
+
+ Reviewed by Alexey Proskuryakov.
+
+ * Configurations/Base.xcconfig:
+ * Configurations/DebugRelease.xcconfig:
+
+2015-01-21 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: support aligned allocation
+ https://bugs.webkit.org/show_bug.cgi?id=140732
+
+ Reviewed by Andreas Kling.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocate): New function for aligned allocation.
+
+ Small and medium requests just allocate and free until they find an
+ aligned pointer. This is slightly inefficient in the worst case, but
+ still constant-time with little-to-no space overhead.
+
+ Large requests use a new API that requires the client to specify both
+ its ideal size and alignment, and the worst-case size you would have to
+ allocate in order to produce some interior pointer of the requested size
+ and alignment. We put the burden of this calculation on the client
+ because it simplifies things if we guarantee that allocation won't fail.
+
+ XLarge requests are easy: we just forward them to vmAllocate, which
+ already supported aligned requests.
+
+ * bmalloc/BoundaryTag.h:
+ * bmalloc/BoundaryTagInlines.h:
+ (bmalloc::BoundaryTag::mergeLeft):
+ (bmalloc::BoundaryTag::mergeRight):
+ (bmalloc::BoundaryTag::merge):
+ (bmalloc::BoundaryTag::deallocate):
+ (bmalloc::BoundaryTag::split):
+ (bmalloc::BoundaryTag::allocate): No behavior change here. I just
+ refactored the interface to remove some reference out parameters in
+ order to clarify what changes and what doesn't.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateXLarge): Added an alignment API.
+
+ (bmalloc::Heap::allocateLarge):
+ * bmalloc/Heap.h: Added an alignment API. I split out allocateLarge into
+ a few variants, so aligned and unaligned allocation could share some code.
+
+ * bmalloc/SegregatedFreeList.cpp:
+ (bmalloc::SegregatedFreeList::take):
+ * bmalloc/SegregatedFreeList.h: Changed to use a separate, explicit API
+ for aligned allocation. It turns out that the aligned path is pretty
+ different, since it ends up searching for two potential ways to satisfy
+ an allocation: either large enough and aligned, or large enough to split
+ into something not aligned and something large enough and aligned.
+
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmAllocate): Switched alignment to come before size because
+ that's how the memalign API specifies it.
+
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateLargeRange): Added an alignment API.
+
+2015-01-20 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: a little bit of cleanup
+ https://bugs.webkit.org/show_bug.cgi?id=140687
+
+ Reviewed by Anders Carlsson.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::isPowerOfTwo): Added a check for 0, since 0 would break a lot
+ of code.
+
+ * bmalloc/BoundaryTag.h:
+ * bmalloc/BoundaryTagInlines.h:
+ (bmalloc::BoundaryTag::mergeLeft):
+ (bmalloc::BoundaryTag::mergeRight):
+ (bmalloc::BoundaryTag::merge):
+ (bmalloc::BoundaryTag::deallocate):
+ (bmalloc::BoundaryTag::split):
+ (bmalloc::BoundaryTag::allocate):
+ (bmalloc::BoundaryTag::mergeLargeLeft): Deleted.
+ (bmalloc::BoundaryTag::mergeLargeRight): Deleted.
+ (bmalloc::BoundaryTag::mergeLarge): Deleted.
+ (bmalloc::BoundaryTag::splitLarge): Deleted. Removed the word "Large"
+ from all these functions, since boundary tags always pertain to large
+ objects, and putting the word "Large" everywhere wasn't helping to
+ explain that.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateXLarge):
+ (bmalloc::Heap::findXLarge):
+ (bmalloc::Heap::deallocateXLarge):
+ * bmalloc/Heap.h:
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateXLarge): Deleted.
+ (bmalloc::VMHeap::findXLarge): Deleted.
+ (bmalloc::VMHeap::deallocateXLarge): Deleted. Moved XLarge allocation
+ from VMHeap to Heap. Since the purpose of the VMHeap is to cache VM
+ ranges, and the VMHeap never caches any XLarge ranges, it doesn't
+ really make sense for the VMHeap to be involved.
+
+2015-01-16 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: refactored XLarge allocation for better alignment
+ https://bugs.webkit.org/show_bug.cgi?id=140582
+
+ Reviewed by Andreas Kling.
+
+ XLarge objects used to be Large objects with an extra bit of metadata
+ that said "actually, I'm not large -- I'm extra large".
+
+ The metadata header in an XLarge allocation made it impossible for the
+ XLarge object to honor a very large alignment request.
+
+ The solution is to stop using a metadata header for XLarge objects, and
+ instead to store explicit metadata on the side.
+
+ This is a bit less astonishing, which is also nice.
+
+ Finding XLarge metadata is now a linear search. That's probably OK, since
+ it was always so in TCMalloc, and the usual number of XLarge allocations
+ in a process is 0.
+
+ This design makes it possible for the heap to cache XLarge allocations
+ with and/or without physical pages. I haven't actually done that yet
+ because the tradeoffs are subtle, so I don't want to do anything without
+ a motivating test case.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate): Removed the concept of an XLargeChunk,
+ since an XLarge allocation is now just a naked buffer without a header.
+
+ (bmalloc::Allocator::allocateXLarge): Added an explicit qualifier for
+ XLarge alignment, since XLargeChunk won't give this to us implicitly
+ anymore.
+
+ * bmalloc/BoundaryTag.h:
+ (bmalloc::BoundaryTag::setRange):
+ (bmalloc::BoundaryTag::isXLarge): Deleted.
+ (bmalloc::BoundaryTag::setXLarge): Deleted.
+ * bmalloc/BoundaryTagInlines.h:
+ (bmalloc::validate):
+ (bmalloc::BoundaryTag::deallocate): Removed the XLarge hacks from Large allocations.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::deallocateXLarge):
+ (bmalloc::Deallocator::deallocateSlowCase):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::findXLarge):
+ (bmalloc::Heap::allocateXLarge):
+ (bmalloc::Heap::deallocateXLarge):
+ * bmalloc/Heap.h: Updated for interface changes.
+
+ * bmalloc/ObjectType.cpp:
+ (bmalloc::objectType):
+ * bmalloc/ObjectType.h:
+ (bmalloc::isXLarge): We can now tell if a pointer is XLarge just by
+ examining its bit pattern -- just like we do for other kinds of
+ allocations -- which is nice.
+
+ * bmalloc/Sizes.h:
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateXLarge):
+ (bmalloc::VMHeap::findXLarge):
+ (bmalloc::VMHeap::deallocateXLarge): Keep an explicit vector of metadata
+ for XLarge allocations.
+
+ * bmalloc/XLargeChunk.h: Removed.
+
+2015-01-16 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: added some infrastructure for aligned allocation
+ https://bugs.webkit.org/show_bug.cgi?id=140572
+
+ Reviewed by Andreas Kling.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::isPowerOfTwo):
+ (bmalloc::roundUpToMultipleOf):
+ (bmalloc::roundDownToMultipleOf): Refactored some duplicate code to use our
+ isPowerOfTwo helper function.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocate):
+ * bmalloc/Allocator.h: Stubbed out an implementation of aligned allocation.
+ Doesn't do anything yet, but does correctly forward to system malloc
+ when bmalloc is disabled.
+
+ * bmalloc/Cache.cpp:
+ (bmalloc::Cache::allocateSlowCaseNullCache):
+ * bmalloc/Cache.h:
+ (bmalloc::Cache::allocate):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::memalign):
+ * bmalloc/mbmalloc.cpp: Stubbed out an API for aligned allocation.
+
+2015-01-13 Geoffrey Garen <ggaren@apple.com>
+
+ Consider alignment when allocating from a SegregatedFreeList
+ https://bugs.webkit.org/show_bug.cgi?id=140408
+
+ Reviewed by Sam Weinig.
+
+ In preparation for supporting aligned allocation.
+
+ No performance change.
+
+ Since this is just one extra branch in an already expensive function,
+ I decided not to duplicate the function just to avoid the branch in
+ the un-aligned case.
+
+ * bmalloc/SegregatedFreeList.cpp:
+ (bmalloc::SegregatedFreeList::take):
+ * bmalloc/SegregatedFreeList.h:
+
+2015-01-13 Geoffrey Garen <ggaren@apple.com>
+
+ Renamed minimum to size in SegregatedFreeList
+ https://bugs.webkit.org/show_bug.cgi?id=140406
+
+ Reviewed by Sam Weinig.
+
+ In preparation for supporting aligned allocation.
+
+ * bmalloc/SegregatedFreeList.cpp:
+ (bmalloc::SegregatedFreeList::takeGreedy):
+ (bmalloc::SegregatedFreeList::take): Every size passed to malloc is
+ really just a minimum. Let's not imply that this value is special.
+
+2015-01-11 Dan Bernstein <mitz@apple.com>
+
+ Geoff is organized, but he is not an organization.
+
+ Rubber-stamped by Anders Carlsson.
+
+ * bmalloc.xcodeproj/project.pbxproj: Removed the ORGANIZATIONNAME project attribute.
+
+2015-01-07 Geoffrey Garen <ggaren@apple.com>
+
+ Make bmalloc work with ASan
+ https://bugs.webkit.org/show_bug.cgi?id=140194
+
+ Reviewed by Mark Lam.
+
+ * bmalloc/BPlatform.h: Added a way to detect Darwin OSes, since we need
+ an OS-specific API to test for loaded runtime libraries.
+
+ * bmalloc/Environment.cpp:
+ (bmalloc::isASanEnabled):
+ (bmalloc::Environment::computeIsBmallocEnabled): Disabled bmalloc if
+ ASan is enabled, since system malloc has the Asan hooks we need.
+
+ You could check for the ASan compile-time flag instead, but doing this
+ check at runtime prepares bmalloc for a world where it is a dynamic
+ library that might be loaded into projects it did not compile with.
+
+2015-01-05 Geoffrey Garen <ggaren@apple.com>
+
+ Fix up bmalloc's PerThread for use on Linux
+ https://bugs.webkit.org/show_bug.cgi?id=139804
+
+ Reviewed by Anders Carlsson.
+
+ The previous implementation was a bit slow.
+
+ * bmalloc/PerThread.h:
+ (bmalloc::PerThreadStorage<Cache>::get):
+ (bmalloc::PerThreadStorage::get):
+ (bmalloc::PerThreadStorage::init): Added a catch-all cross-platform Unix
+ way to do fast per-thread access without taking a lock every time. This
+ probably works on all the platforms we care about, and it matches other
+ techniques we use elsewhere in WebKit.
+
+ (bmalloc::PerThread<T>::getFastCase): Removed the conditional from
+ this class because PerThreadStorage now encapsulates everything that
+ needs to be conditional.
+
+ (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Deleted.
+
+2014-12-26 Dan Bernstein <mitz@apple.com>
+
+ <rdar://problem/19348208> REGRESSION (r177027): iOS builds use the wrong toolchain
+ https://bugs.webkit.org/show_bug.cgi?id=139950
+
+ Reviewed by David Kilzer.
+
+ * Configurations/Base.xcconfig: Only define TOOLCHAINS when building for OS X, doing so
+ in a manner that works with Xcode 5.1.1.
+
+2014-12-15 Geoffrey Garen <ggaren@apple.com>
+
+ Safari crashes when you set Malloc environment variables
+ https://bugs.webkit.org/show_bug.cgi?id=139656
+
+ Reviewed by Michael Saboff.
+
+ I forgot to cover the realloc() case. Whoops. (OoPS?)
+
+ This time around, I ran the full MallocBench test suite in Malloc=1
+ mode, and it passed.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::reallocate):
+ * bmalloc/Allocator.h: Pushed realloc() logic down into the allocator.
+ It needs to be down there so that we can do the short-circuiting check
+ for whether bmalloc is enabled first.
+
+ Also added the check.
+
+ * bmalloc/Cache.cpp:
+ (bmalloc::Cache::scavenge):
+ (bmalloc::Cache::Cache):
+ (bmalloc::Cache::reallocateSlowCaseNullCache):
+ * bmalloc/Cache.h:
+ (bmalloc::Cache::deallocator):
+ (bmalloc::Cache::reallocate): Ditto.
+
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::free):
+ (bmalloc::api::realloc): Ditto.
+
+ (bmalloc::api::scavenge): Pushed this down into Cache to match the
+ surrounding functions.
+
+2014-12-11 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc should support system memory analysis tools (part 2)
+ https://bugs.webkit.org/show_bug.cgi?id=139565
+
+ Reviewed by Mark Lam.
+
+ This patch actually queries the environment to see if memory analysis
+ tools have been enabled.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::scavenge): Don't process the object log if
+ we've disabled bmalloc because it will be full of invalid nullptrs.
+
+ * bmalloc/Environment.cpp:
+ (bmalloc::isMallocEnvironmentVariableSet): Test for the list of known
+ Malloc debugging flags. I also added a plain "Malloc" catch-all for
+ when you want to disable bmalloc without enabling any kind of funny
+ business.
+
+ It would be slightly nicer just to iterate the list of environment
+ variables and strstr them, but getenv is the more portable option,
+ and performance here doesn't really matter.
+
+ (bmalloc::isLibgmallocEnabled): Test for the libgmalloc insertion
+ environment variable.
+
+ (bmalloc::Environment::computeIsBmallocEnabled):
+
+2014-12-11 Geoffrey Garen <ggaren@apple.com>
+
+ Try to fix the iOS simulator build.
+
+ #include the declaration of malloc / free.
+
+ * bmalloc/Allocator.cpp:
+ * bmalloc/Deallocator.cpp:
+
+2014-12-11 Geoffrey Garen <ggaren@apple.com>
+
+ Try to fix the build.
+
+ * bmalloc.xcodeproj/project.pbxproj: Marked a header exported.
+
+2014-12-11 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc should support system memory analysis tools (part 1)
+ https://bugs.webkit.org/show_bug.cgi?id=139559
+
+ Reviewed by Mark Lam.
+
+ This patch adds the hooks to disable bmalloc at runtime if certain
+ environment variables are set, but doesn't actually read from the
+ environment yet.
+
+ No performance change.
+
+ * bmalloc.xcodeproj/project.pbxproj: Added the Environment class, which
+ we'll use to read environment variables and see if memory analysis tools
+ have been enabled.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::Allocator):
+ (bmalloc::Allocator::allocateSlowCase): Added a hook to disable bmalloc
+ on the allocation path. We cache the setting to make the check fast.
+
+ * bmalloc/Allocator.h: Interface changes.
+
+ * bmalloc/Cache.cpp:
+ (bmalloc::Cache::Cache): Pass a heap pointer through to our allocator
+ and deallocator. This main purpose is to enable them to query the
+ environment for whether bmalloc is enabled; but this is also a slightly
+ cleaner way to guarantee to them that the Heap has been pre-initialized.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::Deallocator): If bmalloc is disable, artificially
+ fill the object log to force us to take the slow path on all deallocations.
+
+ (bmalloc::Deallocator::deallocateSlowCase): Do the disabled check.
+
+ * bmalloc/Deallocator.h: Interface changes.
+
+ * bmalloc/Environment.cpp: Added.
+ (bmalloc::Environment::Environment):
+ (bmalloc::Environment::computeIsBmallocEnabled):
+ * bmalloc/Environment.h: Added.
+ (bmalloc::Environment::isBmallocEnabled): This is the class that will
+ encapsulate looking for environment variables that turn on heap
+ analysis tools.
+
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::environment):
+
+ * bmalloc/Mutex.h:
+ (bmalloc::Mutex::Mutex):
+ * bmalloc/StaticMutex.h: A little refactoring to clarify these comments,
+ since I got super confused about them while writing this patch.
+
+ * bmalloc/VMHeap.cpp: Fixed an #include.
+
+2014-12-09 David Kilzer <ddkilzer@apple.com>
+
+ Switch from using PLATFORM_NAME to SDK selectors in ANGLE, bmalloc, gtest, JavaScriptCore, WTF
+ <http://webkit.org/b/139212>
+
+ Reviewed by Joseph Pecoraro.
+
+ * Configurations/Base.xcconfig:
+ - Only set GCC_ENABLE_OBJC_GC, GCC_MODEL_TUNING and TOOLCHAINS
+ on OS X.
+ * Configurations/DebugRelease.xcconfig:
+ - Only set MACOSX_DEPLOYMENT_TARGET and SDKROOT on OS X.
+
+2014-11-07 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc uses 8X more virtual memory than necessary
+ https://bugs.webkit.org/show_bug.cgi?id=138495
+
+ Reviewed by Mark Lam.
+
+ iOS has a per-process virtual memory cap around 1GB, so there's some
+ value to not going totally ham with virtual memory.
+
+ We currently use about 8X the necessary amount:
+ - 2X to align our VM allocation
+ - 4X to reserve small / medium / (2) large chunk VM ranges per superchunk
+
+ We can cut that down:
+ - Return the unaligned portion of our VM allocation (-2X)
+ - Use all the chunks in a superchunk, instead of allocating one
+ chunk per superchunk (-4X)
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::roundUpToMultipleOf): Added a non-constant version of this
+ function so we can call it with getpagesize() at runtime.
+
+ * bmalloc/Chunk.h:
+ * bmalloc/LargeChunk.h:
+ (bmalloc::LargeChunk::create): Deleted. Instead of each chunk allocating
+ its own VM, VMHeap allocates the superchunk and all the chunks in it at a time.
+
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmValidate):
+ (bmalloc::vmAllocate): ASSERT that mmap succeeds to make crashes clearer
+ if it does not succeed. Allocate precisely, and give back the extra.
+
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::allocateSuperChunk):
+ (bmalloc::VMHeap::allocateSmallChunk): Deleted.
+ (bmalloc::VMHeap::allocateMediumChunk): Deleted.
+ (bmalloc::VMHeap::allocateLargeChunk): Deleted. Use all the chunks
+ in a superchunk, instead of just one.
+
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateSmallPage):
+ (bmalloc::VMHeap::allocateMediumPage):
+ (bmalloc::VMHeap::allocateLargeRange):
+ * bmalloc/XLargeChunk.h:
+ (bmalloc::XLargeChunk::create): Updated to match changes above.
+
+2014-11-01 David Kilzer <ddkilzer@apple.com>
+
+ JavaScriptCore is missing debug info for bmalloc because libbmalloc.a is stripped
+ <https://webkit.org/b/138286>
+ <rdar://problem/18847087>
+
+ Reviewed by Dan Bernstein.
+
+ * Configurations/bmalloc.xcconfig: Set STRIP_INSTALLED_PRODUCT
+ to NO for the target that produces libbmalloc.a so that the
+ debug symbols will be linked into JavaScriptCore and end up in
+ its dSYM file.
+
+2014-10-30 Dana Burkart <dburkart@apple.com>
+
+ <rdar://problem/18821260> Prepare for the mysterious future
+
+ Reviewed by Lucas Forschler.
+
+ * Configurations/Base.xcconfig:
+ * Configurations/DebugRelease.xcconfig:
+
+2014-09-24 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: cleaned up fast path vs slow path
+ https://bugs.webkit.org/show_bug.cgi?id=137081
+
+ Reviewed by Sam Weinig.
+
+ Might be a 1% speedup on MallocBench. Also cleans up the code a bit.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::Allocator): Merged the small and medium range
+ caches, just like the small and medium allocators. Ranges are abstract
+ objects that don't really care whether they hold small or medium objects,
+ so they don't need to be segregated.
+
+ (bmalloc::Allocator::scavenge): Ditto.
+
+ (bmalloc::Allocator::allocateBumpRangeSlowCase):
+ (bmalloc::Allocator::allocateBumpRange): Same thing here, except that
+ we do care a tiny bit, because we need to specify small vs medium when
+ allocating new ranges from the heap, to ensure that the heap allocates
+ from the right segment of VM.
+
+ (bmalloc::Allocator::allocateLarge):
+ (bmalloc::Allocator::allocateXLarge): NO_INLINE because this was clouding
+ up the fast path. Large allocation performance is dominated by allocation
+ logic and initialization, so inlining it doesn't help.
+
+ (bmalloc::Allocator::allocateSlowCase): Slow path got a bit cleaner since
+ it doesn't need to distinguish small vs medium objects.
+
+ (bmalloc::Allocator::allocateSmallBumpRange): Deleted.
+ (bmalloc::Allocator::allocateMediumBumpRange): Deleted.
+
+ * bmalloc/Allocator.h:
+ * bmalloc/BumpRange.h:
+
+ * bmalloc/Cache.cpp:
+ (bmalloc::Cache::allocateSlowCase): Deleted.
+ (bmalloc::Cache::deallocateSlowCase): Deleted.
+ * bmalloc/Cache.h:
+ (bmalloc::Cache::allocate):
+ (bmalloc::Cache::deallocate):
+ (bmalloc::Cache::allocateFastCase): Deleted.
+ (bmalloc::Cache::deallocateFastCase): Deleted. Removed the Cache slow
+ paths. The downside to this change is that the fast path branches to two
+ distinct failure cases instead of one. The upside is that the slow path
+ doesn't need to re-read the segment register, which is not as cheap as a
+ normal register, and it doesn't need to do an extra level of function
+ call. Seems to be worth it.
+
+ * bmalloc/Deallocator.h:
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::refillSmallBumpRangeCache):
+ (bmalloc::Heap::refillMediumBumpRangeCache):
+ * bmalloc/Heap.h: Updated for interface changes.
+
+ * bmalloc/Sizes.h: The most ranges a cache will hold is the number of
+ small lines in a page / 2, since any other free lines will coalesce
+ with their neighbors.
+
+2014-09-23 Geoffrey Garen <ggaren@apple.com>
+
+ Rolled out r173346.
+
+ bmalloc should honor the FastMalloc statistics API
+ https://bugs.webkit.org/show_bug.cgi?id=136592
+
+ This didn't really work. Because we allow ranges with and without
+ physical pages to merge, and we allow double-committing and
+ double-decommitting, we can't rely on commit actions to track memory
+ footprint.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::size): Deleted.
+ (bmalloc::Heap::capacity): Deleted.
+ * bmalloc/Heap.h:
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::VMHeap):
+ (bmalloc::VMHeap::allocateSmallChunk):
+ (bmalloc::VMHeap::allocateMediumChunk):
+ (bmalloc::VMHeap::allocateLargeChunk):
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateSmallPage):
+ (bmalloc::VMHeap::allocateMediumPage):
+ (bmalloc::VMHeap::allocateLargeRange):
+ (bmalloc::VMHeap::deallocateSmallPage):
+ (bmalloc::VMHeap::deallocateMediumPage):
+ (bmalloc::VMHeap::deallocateLargeRange):
+ (bmalloc::VMHeap::size): Deleted.
+ (bmalloc::VMHeap::capacity): Deleted.
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::heapSize): Deleted.
+ (bmalloc::api::heapCapacity): Deleted.
+
+2014-09-23 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Allocation should be more precise
+ https://bugs.webkit.org/show_bug.cgi?id=136993
+
+ Reviewed by Gavin Barraclough.
+
+ 13% reduction in heap size on the MallocBench *_memory_warning benchmarks.
+
+ This patch teaches the allocator to merge adjacent free lines into a
+ single allocatable range. This allows us to shrink the size of an
+ individual line without increasing fragmentation or the rate of allocator
+ slow paths.
+
+ We'll only take more slow paths when available memory is sparse, which
+ is exactly when it's worth it. When available memory is dense, we'll
+ take fewer slow paths.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Algorithm.h:
+ (bmalloc::divideRoundingUp):
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::Allocator): Updated for interface changes.
+
+ (bmalloc::Allocator::scavenge): Scavenge by object instead of by line.
+ Now that we merge lines, it's not convenient to scavenge by line.
+
+ (bmalloc::Allocator::allocateSmallBumpRange):
+ (bmalloc::Allocator::allocateMediumBumpRange): Allocate whole ranges
+ instead of individual lines.
+
+ (bmalloc::Allocator::allocateSlowCase):
+ (bmalloc::Allocator::allocateSmallLine): Deleted.
+ (bmalloc::Allocator::allocateMediumLine): Deleted.
+ (bmalloc::Allocator::allocateMedium): Deleted.
+ * bmalloc/Allocator.h:
+ (bmalloc::Allocator::allocateFastCase): Folded medium allocations
+ into the standard fast path with small allocations. Since a BumpAllocator
+ just allocates out of an arbitrary range, it doesn't need to distinguish
+ between small and medium lines.
+
+ * bmalloc/BumpAllocator.h:
+ (bmalloc::BumpAllocator::size):
+ (bmalloc::BumpAllocator::BumpAllocator):
+ (bmalloc::BumpAllocator::init):
+ (bmalloc::BumpAllocator::refill):
+ (bmalloc::BumpAllocator::line): Deleted. No need to track line information
+ anymore: the heap just gives us a pointer and a pre-computed number of
+ objects, and we allocate them.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::processObjectLog): Updated for interface changes.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::initializeLineMetadata): Pre-compute precise metadata
+ detailing where all objects will lie in memory. After we merge two lines,
+ we might allocate an object that spans from one line to the next. This
+ metadata details which bits of memory overlap in that way, and how they
+ overlap.
+
+ (bmalloc::Heap::refillSmallBumpRangeCache):
+ (bmalloc::Heap::refillMediumBumpRangeCache): Scan a whole page at a time,
+ and merge adjacent free lines into BumpRanges.
+
+ (bmalloc::Heap::allocateSmallPage):
+ (bmalloc::Heap::allocateMediumPage):
+ (bmalloc::Heap::deallocateSmallLine):
+ (bmalloc::Heap::deallocateMediumLine): Track pages rather than lines,
+ since we scan for free memory a page at a time.
+
+ (bmalloc::Heap::allocateSmallLineSlowCase): Deleted.
+ (bmalloc::Heap::allocateMediumLineSlowCase): Deleted. Folded into the
+ fast path.
+
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::derefSmallLine):
+ (bmalloc::Heap::derefMediumLine):
+ (bmalloc::Heap::deallocateSmallLine): Deleted.
+ (bmalloc::Heap::allocateSmallLine): Deleted.
+ (bmalloc::Heap::deallocateMediumLine): Deleted.
+ (bmalloc::Heap::allocateMediumLine): Deleted. Updated for interface changes.
+
+ * bmalloc/Line.h:
+ (bmalloc::Line<Traits>::ref):
+ (bmalloc::Line<Traits>::deref):
+ (bmalloc::Line<Traits>::concurrentRef): Deleted. We don't pass a derefCount
+ anymore, since we only ever deref by 1 now.
+
+ * bmalloc/MediumAllocator.h:
+ (bmalloc::MediumAllocator::isNull): Deleted.
+ (bmalloc::MediumAllocator::MediumAllocator): Deleted.
+ (bmalloc::MediumAllocator::line): Deleted.
+ (bmalloc::MediumAllocator::allocate): Deleted.
+ (bmalloc::MediumAllocator::derefCount): Deleted.
+ (bmalloc::MediumAllocator::refill): Deleted.
+ (bmalloc::MediumAllocator::clear): Deleted. Deleted some code that's
+ been dead for a while, since it doesn't build anymore with this patch.
+
+ * bmalloc/Page.h:
+ (bmalloc::Page::sizeClass):
+ (bmalloc::Page::setSizeClass):
+ (bmalloc::Page::smallSizeClass): Deleted.
+ (bmalloc::Page::setSmallSizeClass): Deleted. Renamed setSmallSizeClass
+ to sizeClass, since we use it for medium sizes too.
+
+ * bmalloc/Sizes.h:
+ (bmalloc::Sizes::sizeClass):
+ (bmalloc::Sizes::objectSize): Shrank line sizes to save memory.
+
+ (bmalloc::Sizes::smallSizeClassFor): Deleted.
+ (bmalloc::Sizes::mediumSizeClassFor): Deleted.
+
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::realloc): Now that we have precise objects sizes, realloc
+ can be a bit more precise. It also has to be, since we can't guarantee
+ that an object ends at the end of a line anymore.
+
+2014-09-19 Daniel Bates <dabates@apple.com>
+
+ Always assume internal SDK when building configuration Production
+ https://bugs.webkit.org/show_bug.cgi?id=136925
+ <rdar://problem/18362399>
+
+ Reviewed by Dan Bernstein.
+
+ * Configurations/Base.xcconfig:
+
+2014-09-16 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: moved line caches from the deallocator to the allocator
+ https://bugs.webkit.org/show_bug.cgi?id=136868
+
+ Reviewed by Gavin Barraclough.
+
+ I did this mostly as a simplification, to make it easier to change the
+ allocation strategy.
+
+ No throughput change on MallocBench. Saves about 50kB.
+
+ Since the deallocator needs to lock the heap when freeing lines anyway,
+ there isn't much benefit to giving the deallocator a local cache of
+ deallocated lines.
+
+ We still give the allocator a local cache of lines because that does
+ reduce the frequency at which it needs to lock the heap in order to
+ acquire more lines.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::scavenge):
+ (bmalloc::Allocator::allocateSmallLine):
+ (bmalloc::Allocator::allocateMediumLine):
+ (bmalloc::Allocator::allocateMedium):
+ (bmalloc::Allocator::allocateSlowCase):
+ * bmalloc/Allocator.h:
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::Deallocator):
+ (bmalloc::Deallocator::scavenge):
+ (bmalloc::Deallocator::processObjectLog):
+ (bmalloc::Deallocator::deallocateSmallLine): Deleted.
+ (bmalloc::Deallocator::allocateSmallLine): Deleted.
+ (bmalloc::Deallocator::deallocateMediumLine): Deleted.
+ (bmalloc::Deallocator::allocateMediumLine): Deleted.
+ * bmalloc/Deallocator.h:
+
+ * bmalloc/Sizes.h:
+ * bmalloc/VMAllocate.h: Took the opportunity to make the line cache size
+ exactly one page in size. That's about what we were shooting for anyway,
+ and it may make it easier to switch to per-page allocation in future.
+
+2014-09-15 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: allocate small and medium objects using the same bump pointer class
+ https://bugs.webkit.org/show_bug.cgi?id=136843
+
+ Reviewed by Gavin Barraclough.
+
+ 4% speedup on MallocBench.
+
+ Now that medium-sized objects have dedicated per-size allocators, they
+ don't need to use an arbitrary bump pointer allocator. This means that
+ every allocator knows how many objects it will allocate from the start,
+ and we don't need a post-processing step to adjust refcounts based on
+ real allocation count.
+
+ * bmalloc.xcodeproj/project.pbxproj: Renamed SmallAllocator to BumpAllocator
+ since it's used for small and medium objects now.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::Allocator): Updated to use new interface.
+ (bmalloc::Allocator::scavenge): To "retire" an allocator, we just need
+ to make sure that we finish allocating all the objects in it.
+
+ (bmalloc::Allocator::allocateMedium):
+ (bmalloc::Allocator::allocateSlowCase):
+ (bmalloc::Allocator::retire): Deleted.
+ (bmalloc::Allocator::processSmallAllocatorLog): Deleted.
+ (bmalloc::Allocator::processMediumAllocatorLog): Deleted.
+ * bmalloc/Allocator.h:
+ (bmalloc::Allocator::allocateFastCase): Removed abstractions and data
+ used to post-process an allocator based on how many objects it allocated.
+
+ * bmalloc/BumpAllocator.h: Copied from Source/bmalloc/bmalloc/SmallAllocator.h.
+ (bmalloc::BumpAllocator::BumpAllocator):
+ (bmalloc::BumpAllocator::init):
+ (bmalloc::BumpAllocator::line):
+ (bmalloc::BumpAllocator::validate):
+ (bmalloc::BumpAllocator::allocate):
+ (bmalloc::BumpAllocator::refill):
+ (bmalloc::BumpAllocator::clear): Updated these functions to be agnostic
+ about the kinds of lines they allocate into. In some cases, the line
+ type must be provided as a template parameter by the caller.
+
+ (bmalloc::SmallAllocator::SmallAllocator): Deleted.
+ (bmalloc::SmallAllocator::line): Deleted.
+ (bmalloc::SmallAllocator::allocate): Deleted.
+ (bmalloc::SmallAllocator::objectCount): Deleted.
+ (bmalloc::SmallAllocator::derefCount): Deleted.
+ (bmalloc::SmallAllocator::refill): Deleted.
+ (bmalloc::SmallAllocator::clear): Deleted.
+
+ * bmalloc/ObjectType.h:
+ (bmalloc::isMedium):
+
+ * bmalloc/SmallAllocator.h:
+ (bmalloc::SmallAllocator::isNull): Deleted.
+ (bmalloc::SmallAllocator::canAllocate): Deleted.
+ (bmalloc::SmallAllocator::SmallAllocator): Deleted.
+ (bmalloc::SmallAllocator::line): Deleted.
+ (bmalloc::SmallAllocator::allocate): Deleted.
+ (bmalloc::SmallAllocator::objectCount): Deleted.
+ (bmalloc::SmallAllocator::derefCount): Deleted.
+ (bmalloc::SmallAllocator::refill): Deleted.
+ (bmalloc::SmallAllocator::clear): Deleted.
+
+2014-09-12 Geoffrey Garen <ggaren@apple.com>
+
+ Fixed a goof in bmalloc Vector sizing
+ https://bugs.webkit.org/show_bug.cgi?id=136795
+
+ Reviewed by Gavin Barraclough and Sam Weinig.
+
+ We want our minimum vector to be page-sized since the OS will give us
+ a page no matter what -- but we want that many bytes, and not enough
+ bytes to store that many elements.
+
+ * bmalloc/Vector.h: Math is hard.
+
+2014-09-11 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc should segregate medium-sized objects by line like it does for small-sized objects
+ https://bugs.webkit.org/show_bug.cgi?id=136693
+
+ Reviewed by Gavin Barraclough.
+
+ 4% reduction in heap size on the MallocBench *_memory_warning benchmarks.
+
+ No throughput change.
+
+ We keep an array of medium allocators, just like our array of small
+ allocators.
+
+ In future, we can simplify the allocation fast path by merging the small
+ and medium allocator arrays. For now, this is the simplest change that
+ gets the win.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::Allocator):
+ (bmalloc::Allocator::scavenge):
+ (bmalloc::Allocator::allocateMedium):
+ * bmalloc/Allocator.h:
+ * bmalloc/Sizes.h:
+ (bmalloc::Sizes::mediumSizeClassFor):
+
+2014-09-11 Geoffrey Garen <ggaren@apple.com>
+
+ Reviewed by Sam Weinig.
+
+ Renamed log => retire for clarity.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::scavenge):
+ (bmalloc::Allocator::retire):
+ (bmalloc::Allocator::allocateMedium):
+ (bmalloc::Allocator::allocateSlowCase):
+ (bmalloc::Allocator::log): Deleted.
+ * bmalloc/Allocator.h:
+
+2014-09-11 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: eager scavenge leaves behind a bogus allocator
+ https://bugs.webkit.org/show_bug.cgi?id=136743
+
+ Reviewed by Sam Weinig.
+
+ Be sure to clear the allocator after logging it in the eager scavenge
+ case, so that we don't later try to allocate out of the lines that we
+ have thrown away.
+
+ We didn't need to do this previously because scavenge would only happen
+ at thread exit time, after which no further allocation from the per-thread
+ cache would take place.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::scavenge):
+ * bmalloc/MediumAllocator.h:
+ (bmalloc::MediumAllocator::clear):
+ * bmalloc/SmallAllocator.h:
+ (bmalloc::SmallAllocator::clear):
+
+2014-09-05 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc should honor the FastMalloc statistics API
+ https://bugs.webkit.org/show_bug.cgi?id=136592
+
+ Reviewed by Gavin Barraclough.
+
+ We do this by tracking "size" and "capacity" in the VM heap.
+
+ The VM heap's "capacity" is all the VM we ever allocated.
+
+ The VM heap's "size" the subset of VM currently held onto by the
+ VM heap (and therefore not in use by the regular heap).
+
+ Somewhat ironically, reducing the process's memory footprint, increases
+ the size of the VM heap, since the VM heap holds the pages that are
+ purely virtual and not physical.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::size):
+ (bmalloc::Heap::capacity):
+ * bmalloc/Heap.h:
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::VMHeap):
+ (bmalloc::VMHeap::allocateSmallChunk):
+ (bmalloc::VMHeap::allocateMediumChunk):
+ (bmalloc::VMHeap::allocateLargeChunk):
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::size):
+ (bmalloc::VMHeap::capacity):
+ (bmalloc::VMHeap::allocateSmallPage):
+ (bmalloc::VMHeap::allocateMediumPage):
+ (bmalloc::VMHeap::allocateLargeRange):
+ (bmalloc::VMHeap::deallocateSmallPage):
+ (bmalloc::VMHeap::deallocateMediumPage):
+ (bmalloc::VMHeap::deallocateLargeRange):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::heapSize):
+ (bmalloc::api::heapCapacity):
+
+2014-09-02 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc crashes on the EWS bots (due to bad large object allocation)
+ https://bugs.webkit.org/show_bug.cgi?id=136469
+
+ Reviewed by Andreas Kling.
+
+ It's possible to convince bmalloc to perform a bad large object allocation,
+ through these steps:
+
+ (1) Insert object A into freelist F0.
+
+ (2) Split, merge and split again A's neighbors such that object B is
+ inserted into freelist F0, with boundary tag and size equal to object A,
+ but pointer not completely equal to object A. Put object B at the head of F0.
+
+ (3) Allocate some other object from F0, swapping its position in the
+ freelist with object B, such that object A is now ahead of object B.
+
+ --> Now, the next allocation for size A/B will allocate object A, which
+ has a slightly wrong idea about where the object actually begins.
+ Immediately, you'll corrupt a little memory, and over time, you'll also
+ corrupt boundary tag metadata.
+
+ The solution is to store the begin pointer in the boundary tag. Luckily,
+ this doesn't make the tag any bigger, and it's not a noticeable slowdown
+ on MallocBench.
+
+ * bmalloc/Algorithm.h:
+ (bmalloc::rightShift):
+ * bmalloc/BeginTag.h:
+ (bmalloc::BeginTag::isInFreeList): This is the bug fix. Make sure to
+ validate the start pointer when popping off the free list. Through a
+ very uncommon set of steps, it is possible to have an item in the free
+ list that is valid by all accounts except for its start pointer.
+
+ * bmalloc/BoundaryTag.h:
+ (bmalloc::BoundaryTag::compactBegin):
+ (bmalloc::BoundaryTag::setRange):
+ (bmalloc::BoundaryTag::setSize): Deleted. Record a compact version of the
+ start pointer. We don't need the whole pointer -- just the offset, in
+ largeAlignment increments, into the relevant boundary tag bucket.
+
+ * bmalloc/BoundaryTagInlines.h:
+ (bmalloc::validateNext):
+ (bmalloc::BoundaryTag::init):
+ (bmalloc::BoundaryTag::mergeLarge):
+ (bmalloc::BoundaryTag::splitLarge):
+ * bmalloc/SegregatedFreeList.cpp:
+ (bmalloc::SegregatedFreeList::insert):
+ (bmalloc::SegregatedFreeList::takeGreedy):
+ (bmalloc::SegregatedFreeList::take): Provide the whole range instead of
+ the size when establishing a boundary tag, as required by the new
+ interface.
+
+ * bmalloc/Sizes.h:
+
+2014-08-14 Geoffrey Garen <ggaren@apple.com>
+
+ Fixed a bmalloc crash seen on the EWS bot
+ https://bugs.webkit.org/show_bug.cgi?id=135955
+
+ Reviewed by Andreas Kling.
+
+ * bmalloc/Syscall.h: Some CG APIs vm_copy their input buffers. If the
+ input buffer is a malloc region, that region will get marked Copy-On-Write
+ by the kernel. Calls to madvise() for COW regions fail and return EINVAL
+ on older OS X's. In 10.10, they still fail, but they do not return
+ EINVAL.
+
+ So, we can only ASSERT that our syscalls succeed starting with 10.10.
+
+2014-08-14 Geoffrey Garen <ggaren@apple.com>
+
+ Fixed the bmalloc build
+ https://bugs.webkit.org/show_bug.cgi?id=135953
+
+ Reviewed by Andreas Kling.
+
+ * bmalloc.xcodeproj/project.pbxproj: Marked a few headers as private.
+ These headers are used, so they must be available outside the project.
+
+2014-08-13 Daniel Bates <dabates@apple.com>
+
+ Attempt to fix the build following <http://trac.webkit.org/changeset/172576>
+ (https://bugs.webkit.org/show_bug.cgi?id=135895)
+
+ Substitute PerThreadStorage<T>::initSharedKeyIfNeeded() for initSharedKeyIfNeeded() in
+ implementation of PerThread<T>::getFastCase().
+
+ * bmalloc/PerThread.h:
+ (bmalloc::PerThread<T>::getFastCase):
+
+2014-08-13 Daniel Bates <dabates@apple.com>
+
+ Make bmalloc::PerThread work without C++ thread local storage
+ https://bugs.webkit.org/show_bug.cgi?id=135895
+
+ Reviewed by Geoffrey Garen.
+
+ Implement support for building bmalloc without C++ thread local storage.
+
+ * bmalloc/BPlatform.h: Remove macro define BPLATFORM_IOS_SIMULATOR. Added macro function
+ BCOMPILER_SUPPORTS() and macro define BCOMPILER_SUPPORTS_CXX_THREAD_LOCAL that can be used
+ to determine whether the compiler supports C++ thread local storage.
+ * bmalloc/PerThread.h:
+ (bmalloc::PerThreadStorage::get): Modified to call pthread_getspecific() when building
+ without C++ thread local storage.
+ (bmalloc::PerThreadStorage::initSharedKeyIfNeeded): Added.
+ (bmalloc::PerThreadStorage::init): Moved logic to initialize shared Pthread key from here to
+ PerThreadStorage::initSharedKeyIfNeeded().
+ (bmalloc::PerThread<T>::getFastCase): Modified to call PerThreadStorage::initSharedKeyIfNeeded()
+ before querying PerThreadStorage::get() when building without C++ thread local storage so as to
+ ensure that the shared key has been initialized.
+ (_pthread_setspecific_direct): Deleted.
+ (_pthread_getspecific_direct): Deleted.
+
+2014-08-13 Daniel Bates <dabates@apple.com>
+
+ [iOS] Make JavaScriptCore and bmalloc build with the public SDK
+ https://bugs.webkit.org/show_bug.cgi?id=135848
+
+ Reviewed by Geoffrey Garen.
+
+ * bmalloc/BPlatform.h: Added macro BPLATFORM_IOS_SIMULATOR, which evaluates to true
+ when building for the iOS Simulator.
+ * bmalloc/PerThread.h: Use pthread_machdep.h code path when building for iOS Simulator
+ using the public SDK.
+ (_pthread_setspecific_direct): Added; only defined when building for the iOS Simulator
+ using the public SDK.
+ (_pthread_getspecific_direct): Added; only defined when building for the iOS Simulator
+ using the public SDK.
+
+2014-08-12 Daniel Bates <dabates@apple.com>
+
+ BPLATFORM(IOS) always evaluates to false
+ https://bugs.webkit.org/show_bug.cgi?id=135843
+
+ Reviewed by Geoffrey Garen.
+
+ Fix typo in definition of BPLATFORM() and include system header TargetConditionals.h
+ (when building on an Apple platform) so that BPLATFORM(X) evaluates to true when
+ building for platform X. In particular, so that BPLATFORM(IOS) evaluates to true when
+ building for iOS.
+
+ As a side effect of this change, the change made in <http://trac.webkit.org/changeset/167289>
+ will be honored and iOS will assume a VM page size of 16kB (again) instead of 4kB.
+
+ * bmalloc/BPlatform.h:
+
+2014-08-11 Andy Estes <aestes@apple.com>
+
+ [iOS] Get rid of iOS.xcconfig
+ https://bugs.webkit.org/show_bug.cgi?id=135809
+
+ Reviewed by Joseph Pecoraro.
+
+ All iOS.xcconfig did was include AspenFamily.xcconfig, so there's no need for the indirection.
+
+ * Configurations/Base.xcconfig:
+ * Configurations/iOS.xcconfig: Removed.
+ * bmalloc.xcodeproj/project.pbxproj:
+
+2014-05-01 Dan Bernstein <mitz@apple.com>
+
+ Fixed production builds for the iOS Simulator.
+ <rdar://problem/16792221>
+
+ * Configurations/bmalloc.xcconfig: Include INSTALL_PATH_PREFIX in
+ PRIVATE_HEADERS_FOLDER_PATH when installing.
+
+2014-04-20 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Segregate pages by objects size
+ https://bugs.webkit.org/show_bug.cgi?id=131909
+
+ Reviewed by Andreas Kling.
+
+ 2% reduction in memory-at-end on the Membuster memory_warning benchmarks.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocateSlowCase):
+ * bmalloc/Allocator.h:
+ (bmalloc::Allocator::allocateFastCase):
+ (bmalloc::Allocator::smallAllocatorFor): Use the new shared helper
+ function for size class calculation.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::Deallocator):
+ (bmalloc::Deallocator::scavenge):
+ (bmalloc::Deallocator::deallocateSmallLine):
+ (bmalloc::Deallocator::allocateSmallLine):
+ * bmalloc/Deallocator.h: Keep a cache for every size class, since the
+ cache can't be shared anymore.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateSmallLineSlowCase):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::deallocateSmallLine): Ditto.
+
+ (bmalloc::Heap::allocateSmallLine): Check size class in addition to
+ page refcount when allocating a line because we might have deallocated
+ the page and the recycled it for another size class.
+
+ (bmalloc::Heap::deallocateMediumLine):
+ (bmalloc::Heap::allocateMediumLine):
+ * bmalloc/Line.h:
+ (bmalloc::Line::refCount):
+ * bmalloc/Page.h:
+ (bmalloc::Page::refCount):
+ (bmalloc::Page::smallSizeClass):
+ (bmalloc::Page::setSmallSizeClass):
+ (bmalloc::Page<Traits>::refCount): Deleted.
+ * bmalloc/Sizes.h:
+ (bmalloc::Sizes::smallSizeClassFor): New shared API for computing
+ an index into an array from a size.
+
+2014-04-19 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Improved alignment in LargeChunk
+ https://bugs.webkit.org/show_bug.cgi?id=131895
+
+ Reviewed by Andreas Kling.
+
+ * bmalloc/Chunk.h:
+ * bmalloc/LargeChunk.h: Align to vmPageSize just like Chunk does.
+ Technically, the previous alignment was harmless, but I would prefer,
+ dear reader, not to have to explain the interlocking set of
+ circumstances that made it so.
+
+2014-04-19 Geoffrey Garen <ggaren@apple.com>
+
+ Rolled out r167502 because it caused a crash on the facebook benchmark.
+
+ Unreviewed.
+
+ bmalloc: Added an XSmall line size
+ https://bugs.webkit.org/show_bug.cgi?id=131851
+
+ Reviewed by Sam Weinig.
+
+2014-04-19 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Mutex should be harder to use wrong
+ https://bugs.webkit.org/show_bug.cgi?id=131879
+
+ Reviewed by Andreas Kling.
+
+ Mutex now has a proper constructor, so you can't deadlock by forgetting
+ to initialize it.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::processXSmallAllocatorLog):
+ (bmalloc::Allocator::processSmallAllocatorLog):
+ (bmalloc::Allocator::processMediumAllocatorLog):
+ (bmalloc::Allocator::allocateLarge):
+ (bmalloc::Allocator::allocateXLarge): Global replace Mutex => StaticMutex,
+ since the Heap mutex is a static.
+
+ * bmalloc/AsyncTask.h:
+ (bmalloc::Function>::AsyncTask): Use Mutex, since we're not static. No
+ need for explicit initialization anymore.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::scavenge):
+ (bmalloc::Deallocator::deallocateLarge):
+ (bmalloc::Deallocator::deallocateXLarge):
+ (bmalloc::Deallocator::processObjectLog):
+ (bmalloc::Deallocator::deallocateSmallLine):
+ (bmalloc::Deallocator::deallocateXSmallLine):
+ (bmalloc::Deallocator::allocateSmallLine):
+ (bmalloc::Deallocator::allocateXSmallLine):
+ (bmalloc::Deallocator::deallocateMediumLine):
+ (bmalloc::Deallocator::allocateMediumLine):
+ * bmalloc/Deallocator.h:
+ * bmalloc/Heap.cpp:
+ (bmalloc::sleep):
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::concurrentScavenge):
+ (bmalloc::Heap::scavenge):
+ (bmalloc::Heap::scavengeSmallPages):
+ (bmalloc::Heap::scavengeXSmallPages):
+ (bmalloc::Heap::scavengeMediumPages):
+ (bmalloc::Heap::scavengeLargeRanges):
+ (bmalloc::Heap::allocateXSmallLineSlowCase):
+ (bmalloc::Heap::allocateSmallLineSlowCase):
+ (bmalloc::Heap::allocateMediumLineSlowCase):
+ (bmalloc::Heap::allocateXLarge):
+ (bmalloc::Heap::deallocateXLarge):
+ (bmalloc::Heap::allocateLarge):
+ (bmalloc::Heap::deallocateLarge):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::deallocateXSmallLine):
+ (bmalloc::Heap::allocateXSmallLine):
+ (bmalloc::Heap::deallocateSmallLine):
+ (bmalloc::Heap::allocateSmallLine):
+ (bmalloc::Heap::deallocateMediumLine):
+ (bmalloc::Heap::allocateMediumLine):
+ * bmalloc/Line.h:
+ (bmalloc::Line<Traits>::deref):
+ * bmalloc/Mutex.cpp: Removed.
+ * bmalloc/Mutex.h:
+ (bmalloc::Mutex::Mutex):
+ (bmalloc::Mutex::init): Deleted.
+ (bmalloc::Mutex::try_lock): Deleted.
+ (bmalloc::Mutex::lock): Deleted.
+ (bmalloc::Mutex::unlock): Deleted.
+ * bmalloc/Page.h:
+ (bmalloc::Page<Traits>::ref):
+ (bmalloc::Page<Traits>::deref):
+ (bmalloc::Page<Traits>::refCount):
+ * bmalloc/PerProcess.h:
+ (bmalloc::PerProcess::mutex):
+ (bmalloc::PerProcess<T>::getSlowCase):
+ * bmalloc/StaticMutex.cpp: Added.
+ (bmalloc::StaticMutex::lockSlowCase):
+ * bmalloc/StaticMutex.h: Added.
+ (bmalloc::StaticMutex::init):
+ (bmalloc::StaticMutex::try_lock):
+ (bmalloc::StaticMutex::lock):
+ (bmalloc::StaticMutex::unlock):
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::deallocateXSmallPage):
+ (bmalloc::VMHeap::deallocateSmallPage):
+ (bmalloc::VMHeap::deallocateMediumPage):
+ (bmalloc::VMHeap::deallocateLargeRange):
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::scavenge): Global replace Mutex => StaticMutex,
+ since the Heap mutex is a static.
+
+2014-04-18 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: AsyncTask should use Mutex instead of std::mutex
+ https://bugs.webkit.org/show_bug.cgi?id=131865
+
+ Reviewed by Gavin Barraclough.
+
+ std::mutex is so slow that it makes parallelizing simple tasks through
+ AsyncTask a net regression. Mutex fixes this.
+
+ * bmalloc/AsyncTask.h:
+ (bmalloc::Function>::AsyncTask):
+ (bmalloc::Function>::join):
+ (bmalloc::Function>::runSlowCase):
+ (bmalloc::Function>::entryPoint):
+ * bmalloc/Mutex.h:
+ (bmalloc::Mutex::init):
+
+2014-04-18 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Added an XSmall line size
+ https://bugs.webkit.org/show_bug.cgi?id=131851
+
+ Reviewed by Sam Weinig.
+
+ Reduces malloc footprint on Membuster recordings by 10%.
+
+ This is a throughput regression, but we're still way ahead of TCMalloc.
+ I have some ideas for how to recover the regression -- but I wanted to
+ get this win in first.
+
+ Full set of benchmark results:
+
+ bmalloc> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks --measure-heap nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
+
+ nopatch patch Δ
+ Peak Memory:
+ reddit_memory_warning 7,896kB 7,532kB ^ 1.05x smaller
+ flickr_memory_warning 12,968kB 12,324kB ^ 1.05x smaller
+ theverge_memory_warning 16,672kB 15,200kB ^ 1.1x smaller
+
+ <geometric mean> 11,952kB 11,216kB ^ 1.07x smaller
+ <arithmetic mean> 12,512kB 11,685kB ^ 1.07x smaller
+ <harmonic mean> 11,375kB 10,726kB ^ 1.06x smaller
+
+ Memory at End:
+ reddit_memory_warning 7,320kB 6,856kB ^ 1.07x smaller
+ flickr_memory_warning 10,848kB 9,692kB ^ 1.12x smaller
+ theverge_memory_warning 16,380kB 14,872kB ^ 1.1x smaller
+
+ <geometric mean> 10,916kB 9,961kB ^ 1.1x smaller
+ <arithmetic mean> 11,516kB 10,473kB ^ 1.1x smaller
+ <harmonic mean> 10,350kB 9,485kB ^ 1.09x smaller
+
+ MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
+
+ nopatch patch Δ
+ Execution Time:
+ churn 127ms 151ms ! 1.19x slower
+ list_allocate 130ms 164ms ! 1.26x slower
+ tree_allocate 109ms 127ms ! 1.17x slower
+ tree_churn 115ms 120ms ! 1.04x slower
+ facebook 240ms 259ms ! 1.08x slower
+ fragment 91ms 131ms ! 1.44x slower
+ fragment_iterate 105ms 106ms ! 1.01x slower
+ message_one 260ms 259ms ^ 1.0x faster
+ message_many 149ms 154ms ! 1.03x slower
+ medium 194ms 248ms ! 1.28x slower
+ big 157ms 160ms ! 1.02x slower
+
+ <geometric mean> 144ms 163ms ! 1.13x slower
+ <arithmetic mean> 152ms 171ms ! 1.12x slower
+ <harmonic mean> 137ms 156ms ! 1.14x slower
+
+ MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks nopatch:~/scratch/Build-nopatch/Release/ patch:~/webkit/WebKitBuild/Release/
+
+ nopatch patch Δ
+ Execution Time:
+ churn 126ms 148ms ! 1.17x slower
+ churn --parallel 62ms 76ms ! 1.23x slower
+ list_allocate 130ms 164ms ! 1.26x slower
+ list_allocate --parallel 120ms 175ms ! 1.46x slower
+ tree_allocate 111ms 127ms ! 1.14x slower
+ tree_allocate --parallel 95ms 135ms ! 1.42x slower
+ tree_churn 115ms 124ms ! 1.08x slower
+ tree_churn --parallel 107ms 126ms ! 1.18x slower
+ facebook 240ms 276ms ! 1.15x slower
+ facebook --parallel 802ms 1,088ms ! 1.36x slower
+ fragment 92ms 130ms ! 1.41x slower
+ fragment --parallel 66ms 124ms ! 1.88x slower
+ fragment_iterate 109ms 127ms ! 1.17x slower
+ fragment_iterate --parallel 55ms 64ms ! 1.16x slower
+ message_one 260ms 260ms
+ message_many 170ms 238ms ! 1.4x slower
+ medium 185ms 250ms ! 1.35x slower
+ medium --parallel 210ms 334ms ! 1.59x slower
+ big 150ms 169ms ! 1.13x slower
+ big --parallel 138ms 144ms ! 1.04x slower
+
+ <geometric mean> 135ms 170ms ! 1.26x slower
+ <arithmetic mean> 167ms 214ms ! 1.28x slower
+ <harmonic mean> 117ms 148ms ! 1.26x slower
+
+ MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
+
+ TC patch Δ
+ Peak Memory:
+ reddit_memory_warning 13,836kB 13,436kB ^ 1.03x smaller
+ flickr_memory_warning 24,868kB 25,188kB ! 1.01x bigger
+ theverge_memory_warning 24,504kB 26,636kB ! 1.09x bigger
+
+ <geometric mean> 20,353kB 20,812kB ! 1.02x bigger
+ <arithmetic mean> 21,069kB 21,753kB ! 1.03x bigger
+ <harmonic mean> 19,570kB 19,780kB ! 1.01x bigger
+
+ Memory at End:
+ reddit_memory_warning 8,656kB 10,016kB ! 1.16x bigger
+ flickr_memory_warning 11,844kB 13,784kB ! 1.16x bigger
+ theverge_memory_warning 18,516kB 22,748kB ! 1.23x bigger
+
+ <geometric mean> 12,382kB 14,644kB ! 1.18x bigger
+ <arithmetic mean> 13,005kB 15,516kB ! 1.19x bigger
+ <harmonic mean> 11,813kB 13,867kB ! 1.17x bigger
+
+ MallocBench> ~/webkit/PerformanceTests/MallocBench/run-malloc-benchmarks TC:~/scratch/Build-TCMalloc/Release/ patch:~/webkit/WebKitBuild/Release/
+
+ TC patch Δ
+ Execution Time:
+ churn 416ms 148ms ^ 2.81x faster
+ list_allocate 463ms 164ms ^ 2.82x faster
+ tree_allocate 292ms 127ms ^ 2.3x faster
+ tree_churn 157ms 120ms ^ 1.31x faster
+ facebook 327ms 276ms ^ 1.18x faster
+ fragment 335ms 129ms ^ 2.6x faster
+ fragment_iterate 344ms 108ms ^ 3.19x faster
+ message_one 386ms 258ms ^ 1.5x faster
+ message_many 410ms 154ms ^ 2.66x faster
+ medium 391ms 245ms ^ 1.6x faster
+ big 261ms 167ms ^ 1.56x faster
+
+ <geometric mean> 332ms 164ms ^ 2.02x faster
+ <arithmetic mean> 344ms 172ms ^ 1.99x faster
+ <harmonic mean> 317ms 157ms ^ 2.02x faster
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::Allocator): Don't assume that each allocator's
+ index corresponds with its size. Instead, use the size selection function
+ explicitly. Now that we have XSmall, some small allocator entries are
+ unused.
+
+ (bmalloc::Allocator::scavenge):
+ (bmalloc::Allocator::log):
+ (bmalloc::Allocator::processXSmallAllocatorLog):
+ (bmalloc::Allocator::allocateSlowCase):
+ * bmalloc/Allocator.h:
+ (bmalloc::Allocator::xSmallAllocatorFor):
+ (bmalloc::Allocator::allocateFastCase):
+ * bmalloc/Chunk.h:
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::scavenge):
+ (bmalloc::Deallocator::processObjectLog):
+ (bmalloc::Deallocator::deallocateSlowCase):
+ (bmalloc::Deallocator::deallocateXSmallLine):
+ (bmalloc::Deallocator::allocateXSmallLine):
+ * bmalloc/Deallocator.h:
+ (bmalloc::Deallocator::deallocateFastCase):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::scavenge):
+ (bmalloc::Heap::scavengeXSmallPages):
+ (bmalloc::Heap::allocateXSmallLineSlowCase):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::deallocateXSmallLine):
+ (bmalloc::Heap::allocateXSmallLine):
+ * bmalloc/LargeChunk.h:
+ (bmalloc::LargeChunk::get):
+ (bmalloc::LargeChunk::endTag):
+ * bmalloc/Line.h:
+ * bmalloc/MediumAllocator.h:
+ (bmalloc::MediumAllocator::allocate):
+ (bmalloc::MediumAllocator::refill):
+ * bmalloc/ObjectType.cpp:
+ (bmalloc::objectType):
+ * bmalloc/ObjectType.h:
+ (bmalloc::isXSmall):
+ (bmalloc::isSmall):
+ (bmalloc::isMedium):
+ (bmalloc::isLarge):
+ (bmalloc::isSmallOrMedium): Deleted.
+ * bmalloc/SegregatedFreeList.h: I boiler-plate copied existing code for
+ handling small objects. There's probably a reasonable way to share this
+ code in the future -- I'll look into that once it's stopped changing.
+
+ * bmalloc/Sizes.h: Tweaked size classes to make Membuster happy. This
+ is the main reason things got slower.
+
+ * bmalloc/SmallAllocator.h:
+ (bmalloc::SmallAllocator::allocate):
+ * bmalloc/SmallTraits.h:
+ * bmalloc/VMHeap.cpp:
+ (bmalloc::VMHeap::allocateXSmallChunk):
+ * bmalloc/VMHeap.h:
+ (bmalloc::VMHeap::allocateXSmallPage):
+ (bmalloc::VMHeap::deallocateXSmallPage):
+ * bmalloc/XSmallAllocator.h: Added.
+ (bmalloc::XSmallAllocator::isNull):
+ (bmalloc::XSmallAllocator::canAllocate):
+ (bmalloc::XSmallAllocator::XSmallAllocator):
+ (bmalloc::XSmallAllocator::line):
+ (bmalloc::XSmallAllocator::allocate):
+ (bmalloc::XSmallAllocator::objectCount):
+ (bmalloc::XSmallAllocator::derefCount):
+ (bmalloc::XSmallAllocator::refill):
+ * bmalloc/XSmallChunk.h: Added.
+ * bmalloc/XSmallLine.h: Added.
+ * bmalloc/XSmallPage.h: Added.
+ * bmalloc/XSmallTraits.h: Added.
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::realloc): Boiler-plate copy, as above.
+
+2014-04-14 Geoffrey Garen <ggaren@apple.com>
+
+ MallocBench should scavenge explicitly instead of waiting
+ https://bugs.webkit.org/show_bug.cgi?id=131661
+
+ Reviewed by Andreas Kling.
+
+ Added explicit scavenge support to bmalloc. This isn't a memory win,
+ since bmalloc's per-thread cache is so small. But it makes testing
+ simpler.
+
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::~Allocator):
+ (bmalloc::Allocator::scavenge):
+ * bmalloc/Allocator.h:
+ * bmalloc/Cache.cpp:
+ (bmalloc::Cache::operator new):
+ (bmalloc::Cache::operator delete):
+ (bmalloc::Cache::Cache):
+ (bmalloc::Cache::scavenge):
+ * bmalloc/Cache.h:
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::~Deallocator):
+ (bmalloc::Deallocator::scavenge):
+ * bmalloc/Deallocator.h: Factored existing scavenging code into helper
+ functions, for reuse.
+
+ * bmalloc/Heap.cpp:
+ (bmalloc::sleep):
+ (bmalloc::Heap::concurrentScavenge):
+ (bmalloc::Heap::scavenge):
+ (bmalloc::Heap::scavengeSmallPages):
+ (bmalloc::Heap::scavengeMediumPages):
+ (bmalloc::Heap::scavengeLargeRanges):
+ * bmalloc/Heap.h: Made scavenge sleep duration a parameter. Forced
+ scavenging -- in response to a benchmark or a low memory warning --
+ wants to complete as soon as possible, so its sleep duration is 0.
+
+ * bmalloc/bmalloc.h:
+ (bmalloc::api::scavenge):
+ * bmalloc/mbmalloc.cpp: Exported the scavenge API for MallocBench's use.
+
+2014-04-14 Geoffrey Garen <ggaren@apple.com>
+
+ Use 4kB pages on Mac
+ https://bugs.webkit.org/show_bug.cgi?id=131658
+
+ Reviewed by Sam Weinig.
+
+ This reduces memory use a lot on Membuster:
+
+ base patch Δ
+ Execution Time:
+ reddit_memory_warning 18ms 17ms ^ 1.06x faster
+ flickr_memory_warning 34ms 36ms ! 1.06x slower
+ theverge_memory_warning 39ms 41ms ! 1.05x slower
+
+ <geometric mean> 29ms 29ms ! 1.02x slower
+ <arithmetic mean> 30ms 31ms ! 1.03x slower
+ <harmonic mean> 27ms 27ms ^ 1.0x faster
+
+ Peak Memory:
+ reddit_memory_warning 16,412kB 16,436kB ! 1.0x bigger
+ flickr_memory_warning 30,120kB 30,184kB ! 1.0x bigger
+ theverge_memory_warning 33,408kB 33,420kB ! 1.0x bigger
+
+ <geometric mean> 25,466kB 25,499kB ! 1.0x bigger
+ <arithmetic mean> 26,647kB 26,680kB ! 1.0x bigger
+ <harmonic mean> 24,181kB 24,214kB ! 1.0x bigger
+
+ Memory at End:
+ reddit_memory_warning 2,404kB 1,920kB ^ 1.25x smaller
+ flickr_memory_warning 3,764kB 3,072kB ^ 1.23x smaller
+ theverge_memory_warning 3,648kB 3,132kB ^ 1.16x smaller
+
+ <geometric mean> 3,208kB 2,644kB ^ 1.21x smaller
+ <arithmetic mean> 3,272kB 2,708kB ^ 1.21x smaller
+ <harmonic mean> 3,139kB 2,574kB ^ 1.22x smaller
+
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/BPlatform.h: Added.
+ * bmalloc/VMAllocate.h: Only use 16kB pages on iOS because the page size
+ is 4kB on Mac.
+
+2014-04-14 Alexey Proskuryakov <ap@apple.com>
+
+ Fixed svn:ignore on bmalloc.xcodeproj, it had erroneous leading spaces.
+
+ * bmalloc.xcodeproj: Modified property svn:ignore.
+
+2014-04-13 Geoffrey Garen <ggaren@apple.com>
+
+ Fixed some mbmalloc exports
+ https://bugs.webkit.org/show_bug.cgi?id=131599
+
+ Reviewed by Ryosuke Niwa.
+
+ * bmalloc.xcodeproj/project.pbxproj: Made some headers a private part
+ of the project, so we can call them from API.
+
+ * bmalloc/mbmalloc.cpp: Marked the mbmalloc functions with default
+ visibility, so they show up as exported in the .dylib.
+
+2014-04-09 Geoffrey Garen <ggaren@apple.com>
+
+ Put bmalloc headers in the right place
+ https://bugs.webkit.org/show_bug.cgi?id=131464
+
+ Reviewed by Mark Rowe.
+
+ * Configurations/bmalloc.xcconfig: Set PRIVATE_HEADERS_FOLDER_PATH to
+ specify that we don't just want to dump all of our generically-named
+ headers into /usr/local/include.
+
+2014-04-08 Geoffrey Garen <ggaren@apple.com>
+
+ Made bmalloc more #include friendly
+ https://bugs.webkit.org/show_bug.cgi?id=131386
+
+ Reviewed by Andreas Kling.
+
+ Marked a bunch of headers private so they can be used from client code
+ that #includes bmalloc.h.
+
+ Renamed ASSERT macros to BASSERT. This matches their header, which already
+ had to be renamed, and fixes conflicts with WTF's ASSERT macros.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::allocateSlowCase):
+ * bmalloc/AsyncTask.h:
+ (bmalloc::Function>::runSlowCase):
+ * bmalloc/BAssert.h:
+ * bmalloc/BoundaryTag.h:
+ (bmalloc::BoundaryTag::setSize):
+ * bmalloc/BoundaryTagInlines.h:
+ (bmalloc::validate):
+ (bmalloc::BoundaryTag::init):
+ (bmalloc::BoundaryTag::deallocate):
+ (bmalloc::BoundaryTag::splitLarge):
+ (bmalloc::BoundaryTag::allocate):
+ * bmalloc/Chunk.h:
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::processObjectLog):
+ (bmalloc::Deallocator::deallocateSlowCase):
+ * bmalloc/Deallocator.h:
+ (bmalloc::Deallocator::deallocateFastCase):
+ * bmalloc/FixedVector.h:
+ (bmalloc::Capacity>::operator):
+ (bmalloc::Capacity>::push):
+ (bmalloc::Capacity>::pop):
+ (bmalloc::Capacity>::shrink):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::allocateLarge):
+ * bmalloc/LargeChunk.h:
+ (bmalloc::LargeChunk::get):
+ (bmalloc::LargeChunk::endTag):
+ * bmalloc/Line.h:
+ (bmalloc::Line<Traits>::concurrentRef):
+ (bmalloc::Line<Traits>::deref):
+ * bmalloc/MediumAllocator.h:
+ (bmalloc::MediumAllocator::allocate):
+ * bmalloc/ObjectType.h:
+ (bmalloc::isSmall):
+ * bmalloc/Page.h:
+ (bmalloc::Page<Traits>::ref):
+ (bmalloc::Page<Traits>::deref):
+ * bmalloc/PerThread.h:
+ (bmalloc::PerThread<T>::getSlowCase):
+ * bmalloc/SegregatedFreeList.cpp:
+ (bmalloc::SegregatedFreeList::SegregatedFreeList):
+ (bmalloc::SegregatedFreeList::insert):
+ * bmalloc/SmallAllocator.h:
+ (bmalloc::SmallAllocator::allocate):
+ (bmalloc::SmallAllocator::refill):
+ * bmalloc/Syscall.h:
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmValidate):
+ (bmalloc::vmAllocate):
+ (bmalloc::vmDeallocatePhysicalPagesSloppy):
+ * bmalloc/Vector.h:
+ (bmalloc::Vector<T>::operator):
+ (bmalloc::Vector<T>::pop):
+ (bmalloc::Vector<T>::shrink):
+ * bmalloc/XLargeChunk.h:
+ (bmalloc::XLargeChunk::range):
+ (bmalloc::XLargeChunk::size):
+
+2014-04-08 Geoffrey Garen <ggaren@apple.com>
+
+ Removed an unused file.
+
+ Unreviewed.
+
+ * bmalloc/AsyncTask.cpp: Removed.
+
+2014-04-07 Geoffrey Garen <ggaren@apple.com>
+
+ Build bmalloc on Mac
+ https://bugs.webkit.org/show_bug.cgi?id=131333
+
+ Reviewed by Mark Rowe.
+
+ * Makefile: Added. For make clients.
+
+ These files are required for building any project in WebKit. I copied
+ them from WTF:
+ * Configurations: Added.
+ * Configurations/Base.xcconfig: Added.
+ * Configurations/DebugRelease.xcconfig: Added.
+ * Configurations/bmalloc.xcconfig: Added.
+ * Configurations/iOS.xcconfig: Added.
+ * Configurations/mbmalloc.xcconfig: Added.
+
+ * bmalloc.xcodeproj/project.pbxproj: I removed per-project-file stuff
+ from here because everything is in .xcconfig files now.
+
+ I had to fix a bunch of minor warnings, since they're enabled in our
+ .xcconfig files:
+
+ * bmalloc/AsyncTask.h:
+ (bmalloc::Function>::AsyncTask):
+ * bmalloc/BAssert.h:
+ * bmalloc/BoundaryTagInlines.h:
+ (bmalloc::validate):
+ * bmalloc/Heap.cpp:
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::allocateLarge):
+ (bmalloc::Heap::deallocateLarge):
+ * bmalloc/Mutex.h:
+ (bmalloc::Mutex::Mutex): Deleted.
+ * bmalloc/VMAllocate.h:
+ (bmalloc::vmValidate):
+ * bmalloc/mbmalloc.cpp:
+
+2014-04-07 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: Fixed a leak in the per-thread cache
+ https://bugs.webkit.org/show_bug.cgi?id=131330
+
+ Reviewed by Andreas Kling.
+
+ Remember to deallocate our line caches upon thread exit.
+
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::~Deallocator):
+
+2014-04-07 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc: rolled out the tryLock experiment
+ https://bugs.webkit.org/show_bug.cgi?id=131328
+
+ Reviewed by Andreas Kling.
+
+ It wasn't a speedup.
+
+ * bmalloc.xcodeproj/project.pbxproj:
+ * bmalloc/Allocator.cpp:
+ (bmalloc::Allocator::processSmallAllocatorLog):
+ (bmalloc::Allocator::processMediumAllocatorLog):
+ * bmalloc/Deallocator.cpp:
+ (bmalloc::Deallocator::processObjectLog):
+ (bmalloc::Deallocator::deallocateSlowCase):
+ (bmalloc::Deallocator::deallocateSmallLine):
+ (bmalloc::Deallocator::deallocateMediumLine):
+ * bmalloc/Deallocator.h:
+ (bmalloc::Deallocator::deallocateFastCase):
+ * bmalloc/Heap.h:
+ (bmalloc::Heap::deallocateSmallLine):
+ (bmalloc::Heap::deallocateMediumLine):
+ * bmalloc/Line.h:
+ (bmalloc::Line<Traits>::deref):
+ * bmalloc/Page.h:
+ (bmalloc::Page<Traits>::deref):
+
+2014-04-07 Geoffrey Garen <ggaren@apple.com>
+
+ bmalloc
+ https://bugs.webkit.org/show_bug.cgi?id=131170
+
+ Reviewed by Andreas Kling.
+
+ Initial commit.
+
+ * bmalloc: Added.
+ * bmalloc.xcodeproj: Added.
+ * bmalloc.xcodeproj/project.pbxproj: Added.
+ * bmalloc/Algorithm.h: Added.
+ (bmalloc::max):
+ (bmalloc::min):
+ (bmalloc::mask):
+ (bmalloc::test):
+ (bmalloc::roundUpToMultipleOf):
+ (bmalloc::roundDownToMultipleOf):
+ (bmalloc::sizeOf):
+ (bmalloc::bitCount):
+ (bmalloc::isPowerOfTwo):
+ * bmalloc/Allocator.cpp: Added.
+ (bmalloc::Allocator::Allocator):
+ (bmalloc::Allocator::~Allocator):
+ (bmalloc::Allocator::log):
+ (bmalloc::Allocator::processSmallAllocatorLog):
+ (bmalloc::Allocator::processMediumAllocatorLog):
+ (bmalloc::Allocator::allocateLarge):
+ (bmalloc::Allocator::allocateXLarge):
+ (bmalloc::Allocator::allocateMedium):
+ (bmalloc::Allocator::allocateSlowCase):
+ * bmalloc/Allocator.h: Added.
+ (bmalloc::Allocator::smallAllocatorFor):
+ (bmalloc::Allocator::allocateFastCase):
+ (bmalloc::Allocator::allocate):
+ * bmalloc/AsyncTask.cpp: Added.
+ (bmalloc::AsyncTask<Function>::runSlowCase):
+ (bmalloc::AsyncTask<Function>::pthreadEntryPoint):
+ (bmalloc::AsyncTask<Function>::entryPoint):
+ * bmalloc/AsyncTask.h: Added.
+ (bmalloc::Function>::AsyncTask):
+ (bmalloc::Function>::join):
+ (bmalloc::Function>::run):
+ (bmalloc::Function>::runSlowCase):
+ (bmalloc::Function>::pthreadEntryPoint):
+ (bmalloc::Function>::entryPoint):
+ * bmalloc/BAssert.h: Added.
+ * bmalloc/BeginTag.h: Added.
+ (bmalloc::BeginTag::isInFreeList):
+ * bmalloc/BoundaryTag.h: Added.
+ (bmalloc::BoundaryTag::isXLarge):
+ (bmalloc::BoundaryTag::setXLarge):
+ (bmalloc::BoundaryTag::isFree):
+ (bmalloc::BoundaryTag::setFree):
+ (bmalloc::BoundaryTag::isEnd):
+ (bmalloc::BoundaryTag::setEnd):
+ (bmalloc::BoundaryTag::hasPhysicalPages):
+ (bmalloc::BoundaryTag::setHasPhysicalPages):
+ (bmalloc::BoundaryTag::isNull):
+ (bmalloc::BoundaryTag::clear):
+ (bmalloc::BoundaryTag::size):
+ (bmalloc::BoundaryTag::setSize):
+ (bmalloc::BoundaryTag::prev):
+ (bmalloc::BoundaryTag::next):
+ * bmalloc/BoundaryTagInlines.h: Added.
+ (bmalloc::validate):
+ (bmalloc::validatePrev):
+ (bmalloc::validateNext):
+ (bmalloc::BoundaryTag::init):
+ (bmalloc::BoundaryTag::mergeLargeLeft):
+ (bmalloc::BoundaryTag::mergeLargeRight):
+ (bmalloc::BoundaryTag::mergeLarge):
+ (bmalloc::BoundaryTag::deallocate):
+ (bmalloc::BoundaryTag::splitLarge):
+ (bmalloc::BoundaryTag::allocate):
+ * bmalloc/Cache.cpp: Added.
+ (bmalloc::Cache::operator new):
+ (bmalloc::Cache::operator delete):
+ (bmalloc::Cache::Cache):
+ (bmalloc::Cache::allocateSlowCase):
+ (bmalloc::Cache::allocateSlowCaseNullCache):
+ (bmalloc::Cache::deallocateSlowCase):
+ (bmalloc::Cache::deallocateSlowCaseNullCache):
+ * bmalloc/Cache.h: Added.
+ (bmalloc::Cache::allocator):
+ (bmalloc::Cache::deallocator):
+ (bmalloc::Cache::allocateFastCase):
+ (bmalloc::Cache::deallocateFastCase):
+ (bmalloc::Cache::allocate):
+ (bmalloc::Cache::deallocate):
+ * bmalloc/Chunk.h: Added.
+ (bmalloc::Chunk::begin):
+ (bmalloc::Chunk::end):
+ (bmalloc::Chunk::lines):
+ (bmalloc::Chunk::pages):
+ * bmalloc/Deallocator.cpp: Added.
+ (bmalloc::Deallocator::Deallocator):
+ (bmalloc::Deallocator::~Deallocator):
+ (bmalloc::Deallocator::deallocateLarge):
+ (bmalloc::Deallocator::deallocateXLarge):
+ (bmalloc::Deallocator::processObjectLog):
+ (bmalloc::Deallocator::deallocateSlowCase):
+ (bmalloc::Deallocator::deallocateSmallLine):
+ (bmalloc::Deallocator::allocateSmallLine):
+ (bmalloc::Deallocator::deallocateMediumLine):
+ (bmalloc::Deallocator::allocateMediumLine):
+ * bmalloc/Deallocator.h: Added.
+ (bmalloc::Deallocator::deallocateFastCase):
+ (bmalloc::Deallocator::deallocate):
+ * bmalloc/EndTag.h: Added.
+ (bmalloc::EndTag::operator=):
+ * bmalloc/FixedVector.h: Added.
+ (bmalloc::FixedVector::begin):
+ (bmalloc::FixedVector::end):
+ (bmalloc::FixedVector::size):
+ (bmalloc::FixedVector::capacity):
+ (bmalloc::FixedVector::clear):
+ (bmalloc::FixedVector::isEmpty):
+ (bmalloc::Capacity>::FixedVector):
+ (bmalloc::Capacity>::operator):
+ (bmalloc::Capacity>::push):
+ (bmalloc::Capacity>::pop):
+ (bmalloc::Capacity>::shrink):
+ * bmalloc/Heap.cpp: Added.
+ (bmalloc::sleep):
+ (bmalloc::Heap::Heap):
+ (bmalloc::Heap::concurrentScavenge):
+ (bmalloc::Heap::scavengeSmallPages):
+ (bmalloc::Heap::scavengeMediumPages):
+ (bmalloc::Heap::scavengeLargeRanges):
+ (bmalloc::Heap::allocateSmallLineSlowCase):
+ (bmalloc::Heap::allocateMediumLineSlowCase):
+ (bmalloc::Heap::allocateXLarge):
+ (bmalloc::Heap::deallocateXLarge):
+ (bmalloc::Heap::allocateLarge):
+ (bmalloc::Heap::deallocateLarge):
+ * bmalloc/Heap.h: Added.
+ (bmalloc::Heap::deallocateSmallLine):
+ (bmalloc::Heap::allocateSmallLine):
+ (bmalloc::Heap::deallocateMediumLine):
+ (bmalloc::Heap::allocateMediumLine):
+ * bmalloc/Inline.h: Added.
+ * bmalloc/LargeChunk.h: Added.
+ (bmalloc::LargeChunk::begin):
+ (bmalloc::LargeChunk::end):
+ (bmalloc::LargeChunk::create):
+ (bmalloc::LargeChunk::get):
+ (bmalloc::LargeChunk::beginTag):
+ (bmalloc::LargeChunk::endTag):
+ * bmalloc/Line.h: Added.
+ (bmalloc::Line<Traits>::begin):
+ (bmalloc::Line<Traits>::end):
+ (bmalloc::Line<Traits>::concurrentRef):
+ (bmalloc::Line<Traits>::deref):
+ * bmalloc/MediumAllocator.h: Added.
+ (bmalloc::MediumAllocator::isNull):
+ (bmalloc::MediumAllocator::MediumAllocator):
+ (bmalloc::MediumAllocator::line):
+ (bmalloc::MediumAllocator::allocate):
+ (bmalloc::MediumAllocator::derefCount):
+ (bmalloc::MediumAllocator::refill):
+ * bmalloc/MediumChunk.h: Added.
+ * bmalloc/MediumLine.h: Added.
+ * bmalloc/MediumPage.h: Added.
+ * bmalloc/MediumTraits.h: Added.
+ * bmalloc/Mutex.cpp: Added.
+ (bmalloc::Mutex::lockSlowCase):
+ * bmalloc/Mutex.h: Added.
+ (bmalloc::Mutex::Mutex):
+ (bmalloc::Mutex::try_lock):
+ (bmalloc::Mutex::lock):
+ (bmalloc::Mutex::unlock):
+ * bmalloc/ObjectType.cpp: Added.
+ (bmalloc::objectType):
+ * bmalloc/ObjectType.h: Added.
+ (bmalloc::isSmallOrMedium):
+ (bmalloc::isSmall):
+ * bmalloc/Page.h: Added.
+ (bmalloc::Page<Traits>::ref):
+ (bmalloc::Page<Traits>::deref):
+ (bmalloc::Page<Traits>::refCount):
+ * bmalloc/PerProcess.h: Added.
+ (bmalloc::PerProcess::mutex):
+ (bmalloc::PerProcess<T>::getFastCase):
+ (bmalloc::PerProcess<T>::get):
+ (bmalloc::PerProcess<T>::getSlowCase):
+ * bmalloc/PerThread.h: Added.
+ (bmalloc::PerThreadStorage<Cache>::get):
+ (bmalloc::PerThreadStorage<Cache>::init):
+ (bmalloc::PerThreadStorage::get):
+ (bmalloc::PerThreadStorage::init):
+ (bmalloc::PerThread<T>::getFastCase):
+ (bmalloc::PerThread<T>::get):
+ (bmalloc::PerThread<T>::destructor):
+ (bmalloc::PerThread<T>::getSlowCase):
+ * bmalloc/Range.h: Added.
+ (bmalloc::Range::Range):
+ (bmalloc::Range::begin):
+ (bmalloc::Range::end):
+ (bmalloc::Range::size):
+ (bmalloc::Range::operator!):
+ (bmalloc::Range::operator<):
+ * bmalloc/SegregatedFreeList.cpp: Added.
+ (bmalloc::SegregatedFreeList::SegregatedFreeList):
+ (bmalloc::SegregatedFreeList::insert):
+ (bmalloc::SegregatedFreeList::takeGreedy):
+ (bmalloc::SegregatedFreeList::take):
+ * bmalloc/SegregatedFreeList.h: Added.
+ * bmalloc/Sizes.h: Added.
+ * bmalloc/SmallAllocator.h: Added.
+ (bmalloc::SmallAllocator::isNull):
+ (bmalloc::SmallAllocator::canAllocate):
+ (bmalloc::SmallAllocator::SmallAllocator):
+ (bmalloc::SmallAllocator::line):
+ (bmalloc::SmallAllocator::allocate):
+ (bmalloc::SmallAllocator::objectCount):
+ (bmalloc::SmallAllocator::derefCount):
+ (bmalloc::SmallAllocator::refill):
+ * bmalloc/SmallChunk.h: Added.
+ * bmalloc/SmallLine.h: Added.
+ * bmalloc/SmallPage.h: Added.
+ * bmalloc/SmallTraits.h: Added.
+ * bmalloc/Syscall.h: Added.
+ * bmalloc/VMAllocate.h: Added.
+ (bmalloc::vmSize):
+ (bmalloc::vmValidate):
+ (bmalloc::vmAllocate):
+ (bmalloc::vmDeallocate):
+ (bmalloc::vmDeallocatePhysicalPages):
+ (bmalloc::vmAllocatePhysicalPages):
+ (bmalloc::vmDeallocatePhysicalPagesSloppy):
+ (bmalloc::vmAllocatePhysicalPagesSloppy):
+ * bmalloc/VMHeap.cpp: Added.
+ (bmalloc::VMHeap::VMHeap):
+ (bmalloc::VMHeap::allocateSmallChunk):
+ (bmalloc::VMHeap::allocateMediumChunk):
+ (bmalloc::VMHeap::allocateLargeChunk):
+ * bmalloc/VMHeap.h: Added.
+ (bmalloc::VMHeap::allocateSmallPage):
+ (bmalloc::VMHeap::allocateMediumPage):
+ (bmalloc::VMHeap::allocateLargeRange):
+ (bmalloc::VMHeap::deallocateSmallPage):
+ (bmalloc::VMHeap::deallocateMediumPage):
+ (bmalloc::VMHeap::deallocateLargeRange):
+ * bmalloc/Vector.h: Added.
+ (bmalloc::Vector::begin):
+ (bmalloc::Vector::end):
+ (bmalloc::Vector::size):
+ (bmalloc::Vector::capacity):
+ (bmalloc::Vector::last):
+ (bmalloc::Vector::pop):
+ (bmalloc::Vector<T>::Vector):
+ (bmalloc::Vector<T>::~Vector):
+ (bmalloc::Vector<T>::operator):
+ (bmalloc::Vector<T>::push):
+ (bmalloc::Vector<T>::pop):
+ (bmalloc::Vector<T>::shrink):
+ (bmalloc::Vector<T>::reallocateBuffer):
+ (bmalloc::Vector<T>::shrinkCapacity):
+ (bmalloc::Vector<T>::growCapacity):
+ * bmalloc/XLargeChunk.h: Added.
+ (bmalloc::XLargeChunk::get):
+ (bmalloc::XLargeChunk::begin):
+ (bmalloc::XLargeChunk::XLargeChunk):
+ (bmalloc::XLargeChunk::create):
+ (bmalloc::XLargeChunk::destroy):
+ (bmalloc::XLargeChunk::range):
+ (bmalloc::XLargeChunk::size):
+ * bmalloc/bmalloc.h: Added.
+ (bmalloc::api::malloc):
+ (bmalloc::api::free):
+ (bmalloc::api::realloc):
+ * bmalloc/mbmalloc.cpp: Added.
+