Acknowledgments

저자:

Deji Akomolafe – Staff Solutions Architect, Microsoft Applications Practice Lead
Mark Xu - Sr. Technical Marketing Architect
Oleg Ulyanov - Staff Cloud Solutions Architect

기여해주신 다음 분들께 감사드립니다:

Catherine Xu - Group Manager, Workload Technical Marketing
David Klee - Founder and Chief Architect, Microsoft MVP and vExpert at Heraflux Technologies

[1] Further in the document referenced as SQL Server

[2] Source: vSphere Share Tracker Survey 2017

[3] vSphere vMotion Notifications for Latency Sensitive Applications - https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-vcenter-esxi-management/GUID-0540DF43-9963-4AF9-A4DB-254414DC00DA.html

[4] . For the comprehensive discussion of high availability options refer to https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/solutions/sql-server-on-vmware-availability-and-recovery-options.pdf, https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/solutions/vmware-vsphere-highly-available-mission-critical-sql-server-deployments.pdf

[5] This feature is deprecated as of SQL Server 2012 and should not be used when possible. Consider using Always on Availability Groups instead.

[6] https://cloud.vmware.com/vmc-aws/faq

[7] Licensing considerations are addressed here: https://www.vmware.com/products/vmc-on-aws/microsoft-licensing.html

[8] https://configmax.vmware.com/home

[9] https://www.vmware.com/resources/compatibility/search.php?[10] https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/support/product-lifecycle-matrix.pdf

[11] https://support.microsoft.com/en-us/help/956893/support-policy-for-microsoft-sql-server-products-that-are-running-in-a

[12]https://www.windowsservercatalog.com/results.aspx?&bCatID=1521&cpID=11779&avc=0&ava=136&avt=0&avq=0&OR=1&PGS=25

[13] https://learn.microsoft.com/en-us/troubleshoot/windows-server/virtualization/non-microsoft-hardware-virtualization-software

[14] More details can be found here: https://www.vmware.com/techpapers/2011/understanding-memory-management-in-vmware-vsphere-10206.html

[15] Consult https://docs.vmware.com/en/VMware-vSphere/6.5/vsphere-esxi-vcenter-server-65-availability-guide.pdf for more details.

[16] More details: https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/drs-vsphere65-perf.pdf, https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/vsphere6-drs-perf.pdf

[17] https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vcenterhost.doc/GUID-03E7E5F9-06D9-463F-A64F-D4EC20DAF22E.html

[18] More details: https://docs.vmware.com/en/VMware-vSphere/6.7/vsphere-esxi-vcenter-server-67-resource-management-guide.pdf, chapter 9

[19] Some workloads might benefit from the combination of deep C states for some cores and Turbo boosting another. For this combination, custom BIOS power policy should be used with deep C states enabled and ESXi power policy should be set to “balanced”.

[20] Virtual CPU Configuration and Limitations: https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-vm-administration/GUID-3CDA4DEF-3DE0-4A64-89C7-F31BB77222CB.html

[21] Hyperthreading: https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-resource-management/GUID-FD71CBCA-E97C-4EFA-8A1B-32C09D5DF2A1.html

[22] Intel Hyper-Threading Technology (HT) - https://en.wikipedia.org/wiki/Hyper-threading

[23] See additional information about Hyper-threading on a vSphere host in VMware vSphere Resource Management

[24] It’s recommended to review the publication: http://frankdenneman.nl/2016/07/07/numa-deep-dive-part-1-uma-numa/

[25] The figure is cited from: https://software.intel.com/en-us/articles/optimizing-applications-for-numa

[26] Depending on the implementation and the processor family, this difference could be up to 3X (Source: https://events.static.linuxfound.org/sites/events/files/slides/Optimizing%20Application%20Performance%20in%20Large%20Multi-core%20Systems_0.pdf, p.6.

[27] https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-resource-management/GUID-BD4A462D-5CDC-4483-968B-1DCF103C4208.html

[28] See https://frankdenneman.nl/2022/11/03/vsphere-8-cpu-topology-for-large-memory-footprint-vms-exceeding-numa-boundaries/ for more detailed description and information

[29] https://blogs.msdn.microsoft.com/slavao/2005/08/02/sql-server-2005-numa-support-troubleshooting/

[30] MS SQL Enterprise edition is required to utilize NUMA awareness. https://docs.microsoft.com/en-us/sql/sql-server/editions-and-components-of-sql-server-2016?view=sql-server-2017

[31] Refer to the documentation of the server hardware vendor for more details. Name and value of the setting could be changed or named differently in any particular BIOS/UEFI implementation

[32] If the snooping mode “Cluster-on-die” (CoD, Haswell) or “sub-NUMA cluster” (SNC, Skylake) is used with pCPU with more than 10 cores, each pCPU will be exposed as two logical NUMA nodes (https://software.intel.com/en-us/articles/intel-xeon-processor-scalable-family-technical-overview). VMware ESXi supports CoD starting with vSphere 6.0 and 6.6 U3b (https://kb.vmware.com/s/article/2142499))

[33] https://frankdenneman.nl/numa/

[34] See http://frankdenneman.nl/2016/08/22/numa-deep-dive-part-5-esxi-vmkernel-numa-constructs/ for more details

[35] https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-vm-administration/GUID-EE6F4E5A-3BEA-43DD-9990-DBEB0A280F3A.html?hWord=N4IghgNiBcIKYDcDGIC+Q

[36] Special thanks to V.Bondizo, Sr. Stuff TSE, VMware, for sharing this vmdumper command.

[37] https://docs.microsoft.com/en-us/sysinternals/downloads/coreinfo

[38] For a full list of the supported OSes, check https://learn.microsoft.com/en-us/sql/linux/sql-server-linux-release-notes-2022?view=sql-server-ver16#supported-platforms

[39] Numactl is not available in the standard OS and should be installed using the applicable OS tools or commands

[40] More details can be found here: https://blogs.msdn.microsoft.com/bobsql/2016/06/03/sql-2016-it-just-runs-faster-automatic-soft-numa/ and https://docs.microsoft.com/en-us/sql/database-engine/configure-windows/soft-numa-sql-server?view=sql-server-2017

[41] https://blogs.msdn.microsoft.com/psssql/2010/04/02/how-it-works-soft-numa-io-completion-thread-lazy-writer-workers-and-memory-nodes/

[42] https://docs.microsoft.com/en-us/sql/sql-server/compute-capacity-limits-by-edition-of-sql-server?view=sql-server-2017

[43] More details: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.vm_admin.doc/GUID-EE6F4E5A-3BEA-43DD-9990-DBEB0A280F3A.html

[44] More details and architecture recommendation for SQL Server can be found here: https://docs.microsoft.com/en-us/sql/relational-databases/memory-management-architecture-guide?view=sql-server-2017

[45] Following resource is highly recommended for deeper understanding of memory management in ESXi https://www.vmware.com/techpapers/2011/understanding-memory-management-in-vmware-vsphere-10206.html

[46] Understanding Memory Overhead

[47] More details on the unswap command can be found here: http://www.yellow-bricks.com/2016/06/02/memory-pages-swapped-can-unswap/ Bear in mind that this command is still officially not supported.

[48] VMware Tools must be installed on the guest; status of the tool service must be running and balloon driver must not be disabled

[49] To rebalance memory between vNUMA nodes, a VM should be powered off or moved with a vMotion to a different host.

[50] See https://docs.vmware.com/en/VMware-vSphere/6.7/vsphere-esxi-vcenter-server-67-resource-management-guide.pdf for more details

[51] VM hardware version 14 and a guest OS that supports NVM technology must be used

[52] More details can be found here: https://blogs.msdn.microsoft.com/sqlserverstorageengine/2016/12/02/transaction-commit-latency-acceleration-using-storage-class-memory-in-windows-server-2016sql-server-2016-sp1/ and https://blogs.msdn.microsoft.com/bobsql/2016/11/08/how-it-works-it-just-runs-faster-non-volatile-memory-sql-server-tail-of-log-caching-on-nvdimm/

[53] Size of PMem device should be at least 100 MB to allow creation of GPT partition, which is the prerequisites of the DAX enabled SCM

[54] https://www.youtube.com/watch?v=jDMt9UWAPJQ&feature=youtu.be

[55] More details: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.storage.doc/GUID-9282A8E0-2A93-4F9E-AEFB-952C8DCB243C.html

[56] More details: https://support.microsoft.com/en-us/help/304261/description-of-support-for-network-database-files-in-sql-server

[57] More details: https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/performance/sql-server-vsphere65-perf.pdf, p.10-11

[58] For more details, see Performance Characterization of VMFS and RDM Using a SAN (

https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/performance_char_vmfs_rdm.pdf)

[59] https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-storage/GUID-710B8E1E-9B33-4229-B974-C51A316C3256.html?hWord=N4IghgNiBcIMYQK4GcAuBTATugJgAgDcBbHAaxAF8g

[60]https://configmax.esp.vmware.com/guest?vmwareproduct=vSphere&release=vSphere%208.0&categories=1-0

[61] https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-windows-server-failover/GUID-97B054E2-2EB0-4E10-855B-521A38776F39.html

[62] https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-windows-server-failover/GUID-04626D3C-A305-40BE-A7B9-4E7C7A30BA3D.html

[63] More details can be obtained here: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.storage.doc/GUID-EE1BD912-03E7-407D-8FDC-7F596E41A8D3.html and https://www.vmware.com/files/pdf/products/virtualvolumes/VMware-Whats-New-vSphere-Virtual-Volumes.pdf

[64] Consult your storage array vendor for the recommended firmware version for the full VAAI support.

[65] More details: https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.resmgmt.doc/GUID-DBCAA3F6-D54A-41DA-ACFC-57CCB7E8DF2A.html

[66] Consult https://configmax.vmware.com/guest for more details

[67] https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.vm_admin.doc/GUID-63E09187-0E75-405B-97C7-B48DA1B1734F.html

[68] https://kb.vmware.com/s/article/1015180

[69] https://blogs.vmware.com/kb/2010/06/vmware-snapshots.html

[70] https://kb.vmware.com/kb/1013163

[71] http://kb.vmware.com/kb/1002836, https://cormachogan.com/2015/04/28/when-and-why-do-we-stun-a-virtual-machine/

[72] Depending on a workload and an environment this recommendation may vary, but in general should not exceed 72 hours

[73] More technical materials on SQL Server on vSAN can be found here. For more details: https://core.vmware.com/business-critical-application-reference-architectures

[74] https://blogs.vmware.com/virtualblocks/2018/04/17/whats-new-vmware-vsan-6-7/ and https://storagehub.vmware.com/t/vmware-vsan/sql-server-fci-and-file-server-on-vmware-vsan-6-7-using-iscsi-service/

[75] https://core.vmware.com/resource/sql-server-failover-cluster-instance-vmware-vsan-native

[76] https://blogs.vmware.com/virtualblocks/2022/08/30/announcing-vsan-8-with-vsan-express-storage-architecture/

[77] For the latest performance study of VM encryption, see the following paper: http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/vm-encryption-vsphere65-perf.pdf.

[78] Consult this document for a full description: https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-esxi-vcenter-80-security-guide.pdf

[79] More details https://docs.microsoft.com/en-us/windows-hardware/design/device-experiences/oem-vbs and here https://blogs.msdn.microsoft.com/sqlsecurity/2017/10/05/enabling-confidential-computing-with-always-encrypted-using-enclaves-early-access-preview/

[80] More details: https://docs.vmware.com/en/VMware-Tools/

[81] https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.vm_admin.doc/GUID-64D4B1C9-CD5D-4C68-8B50-585F6A87EBA0.html

[82] https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.vm_admin.doc/GUID-789C3913-1053-4850-A0F0-E29C3D32B6DA.html

[83] https://kb.vmware.com/s/article/1010675

[84] Consult the document for more details: https://blogs.msdn.microsoft.com/docast/2018/02/01/operating-system-best-practice-configurations-for-sql-server/

[85] https://support.microsoft.com/en-au/help/2207548/slow-performance-on-windows-server-when-using-the-balanced-power-plan

[86] https://support.microsoft.com/en-au/help/2207548/slow-performance-on-windows-server-when-using-the-balanced-power-plan

[87] Starting with the VMware Tools version 10.2.5, RSS is enabled by default for VMXNET3 adapter for the new installation of tools. If VMware Tools that were upgraded from lower version than 10.2.5, steps listed in this document are required in order to ensure or confirm that RSS is enabled on the network card.

[88] http://kb.vmware.com/kb/1010398

[89] http://kb.vmware.com/kb/2053145

[90] https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.troubleshooting.doc/GUID-53B382A8-0330-47C1-8E43-94125BCA8AD0.html

[91] See KB309422, How to choose antivirus software to run on computers that are running SQL Server.

[92] http://msdn.microsoft.com/en-us/library/ms190730.aspx

[93] Refer to SQL Server and Large Pages Explained (

https://techcommunity.microsoft.com/t5/sql-server-support-blog/sql-server-and-large-pages-explained-8230/ba-p/315787) for additional information on running SQL Server with large pages.

[94] http://www.vmware.com/resources/techresources/1039

[95] https://docs.vmware.com/en/VMware-vRealize-True-Visibility-Suite/1.0/microsoft-sql-server/GUID-21C00D61-5E63-4723-AD2A-56E8D3971C47.html