Highlighted
hersh1
2 Iron

Confirm Data on SSD Strategy is working

Jump to solution

I've set a directory to use Data on SSD for its SSD strategy however when running a test workload all the disk activity still occurs on the HDDs. I'm guessing I'm missing a setting somewhere. About this cluster/node pool:

S210 node pool - 4 nodes total

L3 Cache was enabled it is now disabled (same results after disabling)

smartpools is NOT licensed

Directory is set to Manually managed - Data on SSD.

Any thoughts on what could be wrong?

Labels (1)
Tags (3)
0 Kudos
1 Solution

Accepted Solutions
hersh1
2 Iron

Re: Confirm Data on SSD Strategy is working

Jump to solution

Looks like all I needed to do was ask the question in a public forum before I realized what was wrong. I disabled L3 cache under Smartpool settings. I needed to disable it for the nodepool.

Before:

Cluster-1# isi storagepool nodepools view --name=s210_18tb_3.2tb-ssd_256gb

                 ID: 2

               Name: s210_18tb_3.2tb-ssd_256gb

              Nodes: 1, 2, 3, 4

  Protection Policy: +2:1

             Manual: No

         L3 Enabled: Yes

L3 Migration Status: l3

After:

Cluster-1# isi storagepool nodepools view --name=s210_18tb_3.2tb-ssd_256gb

                 ID: 2

               Name: s210_18tb_3.2tb-ssd_256gb

              Nodes: 1, 2, 3, 4

  Protection Policy: +2:1

             Manual: No

         L3 Enabled: No

L3 Migration Status: storage

0 Kudos
8 Replies
hersh1
2 Iron

Re: Confirm Data on SSD Strategy is working

Jump to solution

Looks like all I needed to do was ask the question in a public forum before I realized what was wrong. I disabled L3 cache under Smartpool settings. I needed to disable it for the nodepool.

Before:

Cluster-1# isi storagepool nodepools view --name=s210_18tb_3.2tb-ssd_256gb

                 ID: 2

               Name: s210_18tb_3.2tb-ssd_256gb

              Nodes: 1, 2, 3, 4

  Protection Policy: +2:1

             Manual: No

         L3 Enabled: Yes

L3 Migration Status: l3

After:

Cluster-1# isi storagepool nodepools view --name=s210_18tb_3.2tb-ssd_256gb

                 ID: 2

               Name: s210_18tb_3.2tb-ssd_256gb

              Nodes: 1, 2, 3, 4

  Protection Policy: +2:1

             Manual: No

         L3 Enabled: No

L3 Migration Status: storage

0 Kudos
kipcranford
2 Iron

Re: Confirm Data on SSD Strategy is working

Jump to solution

You can always spot-check your files to see where they live by doing this:

# cd /ifs/data/testdir

# isi get -DD <test file> | less

Scroll down a bit until you see a section  like this:

PROTECTION GROUPS

        lbn 0: 4+2/2

                2,9,190593433600:8192#13

                2,9,190594203648:8192#3

                2,31,190529372160:8192#16

                3,29,190239244288:8192#2

                3,29,190238343168:8192#14

                3,9,190377672704:8192#16

                1,28,190292574208:8192#16

                1,29,190211964928:8192#14

                1,29,190211686400:8192#2

These "lbn" portions are your file stripes (they're called "protection groups" internally).  The syntax is:

<node ID>,<drive LNUM>,<address>:<block size>#<number of contiguous blocks in stripe unit>

Just compare the LNUM from "isi devices" (SSD will be at the top of list, then HDD after) to the LNUM in this output to ensure that the blocks are on the media you think they should be...

Peter_Sero
3 Zinc

Re: Confirm Data on SSD Strategy is working

Jump to solution

What does   0,0,0:8192#27 mean?

PROTECTION GROUPS

  lbn 0: 4+2/2

  12,1,355268247552:8192#1

  12,1,355268263936:8192#6

  12,1,355270082560:8192#4

  12,1,370802941952:8192#1

  12,1,370803384320:8192#4

  12,10,472858992640:8192#16

  11,2,477834403840:8192#5

  0,0,0:8192#27

  13,1,376288796672:8192#10

  13,1,376289812480:8192#6

  13,17,479887794176:8192#11

  13,17,623156789248:8192#5

0 Kudos
kipcranford
2 Iron

Re: Confirm Data on SSD Strategy is working

Jump to solution

It's a sparse region within that protection group / stripe.  If that's the only PG for your file, then that likely indicates an area past the end of file.  In other words, it represents where something would have been had the PG been completely full.  The 0,0,0.. construct doesn't indicate any allocation, it's more just an indicator of a logical construct in that advanced output...

Peter_Sero
3 Zinc

Re: Confirm Data on SSD Strategy is working

Jump to solution

Thank you! "no allocation" is what I take home here.

0 Kudos
kipcranford
2 Iron

Re: Confirm Data on SSD Strategy is working

Jump to solution

Correct, no allocation.

As an example, on my 3 node cluster at +2:1 (so stripes of 4+2/2), I create a 480 KiB file, which consists of 3 full stripe units (PGs) and a remaining, partially full stripe unit of 96 KiB (or 12 blocks).  The remaining, empty space in that stripe unit, 32 KiB or 4 blocks, is shown in the "isi get -DD" output as a sparse region (0,0,0:8192#4):

PROTECTION GROUPS

        lbn 0: 4+2/2

                2,10,143425536:8192#16

                2,12,276766720:8192#7

                2,12,75497472:8192#9

                1,31,213786624:8192#16

                1,10,176218112:8192#12

                0,0,0:8192#4

                3,31,110108672:8192#16

                3,13,143253504:8192#16

And just to reiterate, any stripe units in OneFS that are not completely full, DO NOT GET ALLOCATED.  Only the blocks that have file data in them get allocated...

Peter_Sero
3 Zinc

Re: Confirm Data on SSD Strategy is working

Jump to solution

Super! Thanks again!

0 Kudos
hersh1
2 Iron

Re: Confirm Data on SSD Strategy is working

Jump to solution

Thanks for added info, I remembered that get -DD command from training but couldn't remember what it was telling me. Just tested it out and I can confirm the files are now being stored on the expected drive type.

Thanks!

0 Kudos