Not necessarially. If the user selects an item in the right list and one or more in the left list, then the items from the left list are inserted below the selected item in the left list.

In your example of swaping items 3 & 4 in the right list, it becomes a two step process.

1. Select item 3 in right list, click "move left" button.

   The screen is re-drawn with item from position 3 at the bottom of the left hand list.

2. Select the bottom item in the left list, select the third item in the right list, click "move right".

   The list is re-drawn with the previously third item moved below the previously 4th item.

   The rest of the list remains in situ.

It's actually much harder to describe than do, but I guess that's one reason for not using it. Indicating how to use it to your average (impatient) user.

You can also code the logic of the switching from one to the other into javascript associated with the buttons if server hits is a concern, and you can guarentee your users will have js turned on. You could even get clever and have the buttons use javascript if available or submit to the server if not. Just a thought:)