Project

General

Profile

Actions

Feature #2087

closed

lightweight filestore workload generator

Added by Sage Weil about 12 years ago. Updated about 12 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Joao Eduardo Luis
Category:
-
Target version:
% Done:

0%

Source:
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Description

simple program that uses FileStore and generates something that looks vaguely like what an OSD does. e.g.,

- stream of transactions with something like
- small(or large) write to new object
- setxattr on that object
- append to per-collection log object
- set collection xattr

do that with say 50 parallel streams to different collections.

Actions #1

Updated by Sage Weil about 12 years ago

  • Target version changed from v0.43 to v0.44
Actions #2

Updated by Sage Weil about 12 years ago

  • Translation missing: en.field_position set to 1138
Actions #3

Updated by Sage Weil about 12 years ago

  • Translation missing: en.field_story_points set to 5
  • Translation missing: en.field_position deleted (1138)
  • Translation missing: en.field_position set to 1138
Actions #4

Updated by Sage Weil about 12 years ago

  • Assignee set to Samuel Just
Actions #5

Updated by Samuel Just about 12 years ago

  • Assignee changed from Samuel Just to Joao Eduardo Luis

Mark suggested that this would be a good talk for Joao.

Actions #6

Updated by Samuel Just about 12 years ago

*task

Actions #7

Updated by Sage Weil about 12 years ago

  • Status changed from New to In Progress
Actions #8

Updated by Sage Weil about 12 years ago

  • Target version changed from v0.44 to v0.45
  • Translation missing: en.field_position deleted (1150)
  • Translation missing: en.field_position set to 8
Actions #9

Updated by Sage Weil about 12 years ago

  • Translation missing: en.field_position deleted (13)
  • Translation missing: en.field_position set to 12
Actions #10

Updated by Sage Weil about 12 years ago

  • Translation missing: en.field_position deleted (10)
  • Translation missing: en.field_position set to 1
  • Translation missing: en.field_position changed from 1 to 1167
Actions #11

Updated by Joao Eduardo Luis about 12 years ago

  • Status changed from In Progress to 4

Pushed a working version to ceph's git repository, branch wip-2087 [1]. Feedback would be appreciated.

[1] - https://github.com/ceph/ceph/commit/6158e0ccb48b0598cffdbf3333a9c38afc78d5ae

Actions #12

Updated by Joao Eduardo Luis about 12 years ago

Pushed a new commit to [1], making the code compliant with the CodeStyle and with Sage's suggestions on github.

[1] - https://github.com/ceph/ceph/commit/8aa0e798160e4400bec0d9f975c1f2fb4b3e1ba2

Actions #13

Updated by Joao Eduardo Luis about 12 years ago

  • Status changed from 4 to 7
Actions #14

Updated by Joao Eduardo Luis about 12 years ago

  • Status changed from 7 to In Progress

Looks like some memory should be leaking bad, such that valgrind hangs on exit.

19080
19080 HEAP SUMMARY:
19080 in use at exit: 9,454,160,343 bytes in 68,984 blocks
19080 total heap usage: 973,116 allocs, 904,132 frees, 19,001,241,545 bytes allocated
19080
^C^C^C^C^C^C
Actions #15

Updated by Joao Eduardo Luis about 12 years ago

  • Status changed from In Progress to 7

Memory leak fixed.

Apparently, the FileStore does not cleanup after transactions once they are applied, which may lead to huge memory leaks.

Now we simply 'delete m_tx' in the transaction's callback class.

Valgrind still reports memory leaks, but they seem to come from the FileStore class instead of being directly related with our WorkloadGenerator class.

Actions #16

Updated by Sage Weil about 12 years ago

  • Status changed from 7 to Resolved
Actions

Also available in: Atom PDF