Author Topic: FTP Script that only sends half the files  (Read 1312 times)

VastOne

  •      v-ger
  • Posts: 3794
    • View Profile
    • VSIDO Community
FTP Script that only sends half the files
« on: February 22, 2016, 08:19:42 PM »
Having issues with filezilla and TLS connections killing all uploads, I scrounged around and found a workable script for uploading the ISO's and md5 files:

This is /usr/bin/upl.sh and it is set as an executable

Code: [Select]
#!/bin/sh
HOST='my_ftp_site'
USER='my_login'
PASSWD='my_password'
FILE='*'

ftp -inv $HOST <<END_SCRIPT
quote USER $USER
quote PASS $PASSWD
lcd ~/host/upload
binary
prompt off
mput $FILE
bye
##quit
END_SCRIPT
##exit 0

The files are in ~/host/upload and they are going to the root (/) of my directory space on the server

The issue I have is for whatever reasons it uploads x32 iso and md5 perfectly but then stops and kicks out on the x64 files... If I remove the x32 and restart it, the x64 go up perfectly

Anyone see the issue?  Makes very little sense to me since the variable * is for all files and the script sees the x64 files, it just does not send them

This is the terminal output

Code: [Select]
vastone@vsido:~/host/upload$ upl.sh
Connected to ftp.nixnut.com.
220 64.14.73.29 FTP server ready
331 Password required for vsido
230 User vsido logged in
Local directory now /home/vastone/host/upload
200 Type set to I
Interactive mode on.
mput VSIDO_x32_22Feb2016.iso? 200 PORT command successful
150 Opening BINARY mode data connection for VSIDO_x32_22Feb2016.iso
226 Transfer complete
635437056 bytes sent in 1345.74 secs (461.1177 kB/s)
mput VSIDO_x32_22Feb2016.iso.md5? 200 PORT command successful
150 Opening BINARY mode data connection for VSIDO_x32_22Feb2016.iso.md5
226 Transfer complete
58 bytes sent in 0.00 secs (1.3828 MB/s)
mput VSIDO_x64_22Feb2016.iso? mput VSIDO_x64_22Feb2016.iso.md5? 221 Goodbye.
VSIDO      VSIDO Twitter     VSIDO FB     VSIDO Google+   

I dev VSIDO

PackRat

  • Posts: 1771
    • View Profile
Re: FTP Script that only sends half the files
« Reply #1 on: February 22, 2016, 09:23:15 PM »
I think you may have a conflict with your interactive mode -

Code: [Select]
ftp -i
starts ftp with interactive mode off then you have

Code: [Select]
prompt off
I don't think prompt requires or recognizes on/off arguments, it's more like a toggle for interactive mode (see the ftp man page)

Edit - and I just spotted "Interactive mode on" in your terminal output.

visual aid -



So you started with interactive mode disabled, then turned it on with prompt; the server timed out without interaction - but I think it should have asked you if you wanted to upload additional files? Maybe not with mput?

You can try commenting out the "prompt off" line and see if uploads go to completion.

Also, can you test your script as written by uploading 4 small text files with completely different names?
I am tired of talk that comes to nothing.
-- Chief Joseph

...the sun, the darkness, the winds are all listening to what we have to say.
-- Geronimo

VastOne

  •      v-ger
  • Posts: 3794
    • View Profile
    • VSIDO Community
Re: FTP Script that only sends half the files
« Reply #2 on: February 22, 2016, 09:32:30 PM »
You nailed it with the prompt off statement

I tried sending 7 small different files and every time it stopped after the second one.  I removed prompt off and all 7 went through perfectly

Later tonight I will delete the latest ISO's from the server and re up the files to confirm it all works... (Not currently ready to burn all the bandwidth to do it again)

Thanks RatMan, Have a cigar You're gonna go far, you're gonna fly     8)


VSIDO      VSIDO Twitter     VSIDO FB     VSIDO Google+   

I dev VSIDO

PackRat

  • Posts: 1771
    • View Profile
Re: FTP Script that only sends half the files
« Reply #3 on: February 22, 2016, 09:36:23 PM »
You're gonna make it if you try
They're going to love you

Shine on you crazy diamond; keep cranking out a quality distro.
I am tired of talk that comes to nothing.
-- Chief Joseph

...the sun, the darkness, the winds are all listening to what we have to say.
-- Geronimo

VastOne

  •      v-ger
  • Posts: 3794
    • View Profile
    • VSIDO Community
Re: FTP Script that only sends half the files
« Reply #4 on: February 23, 2016, 04:15:44 AM »
The script is now functioning by including everything but it has stalled at about 94% of the second ISO (x64) and is not moving

This may be some sort of data / port open time/size gap... not sure

Edit - Long story short, the shortcomings of FTP is the real issue
VSIDO      VSIDO Twitter     VSIDO FB     VSIDO Google+   

I dev VSIDO

PackRat

  • Posts: 1771
    • View Profile
Re: FTP Script that only sends half the files
« Reply #5 on: February 23, 2016, 03:02:49 PM »
Did a quick search - a lot of inks that point to changing the mtu of the client network interface. You can research that and see if it's something you want to try.
I am tired of talk that comes to nothing.
-- Chief Joseph

...the sun, the darkness, the winds are all listening to what we have to say.
-- Geronimo

PackRat

  • Posts: 1771
    • View Profile
Re: FTP Script that only sends half the files
« Reply #6 on: February 23, 2016, 08:56:21 PM »
Can you use rsync for this procedure?
I am tired of talk that comes to nothing.
-- Chief Joseph

...the sun, the darkness, the winds are all listening to what we have to say.
-- Geronimo

VastOne

  •      v-ger
  • Posts: 3794
    • View Profile
    • VSIDO Community
Re: FTP Script that only sends half the files
« Reply #7 on: February 23, 2016, 08:57:34 PM »
^ Yes.. I think so and even curl

Will test and let you know
VSIDO      VSIDO Twitter     VSIDO FB     VSIDO Google+   

I dev VSIDO

PackRat

  • Posts: 1771
    • View Profile
Re: FTP Script that only sends half the files
« Reply #8 on: February 23, 2016, 10:16:46 PM »
The other option may be to compress the iso files.

For what it's worth, I downloaded all 4 files then used both ftp manually and your script to upload them to the ftp server I run on the desktop computer. Both test uploads went to completion and the md5sums checked out.
I am tired of talk that comes to nothing.
-- Chief Joseph

...the sun, the darkness, the winds are all listening to what we have to say.
-- Geronimo

jedi

  • DEFCON 1
  • Posts: 1474
  • Not all those who wander are lost.
    • View Profile
    • NixNut.com
Re: FTP Script that only sends half the files
« Reply #9 on: February 24, 2016, 06:37:12 AM »
Hey all, I can (and will, if needed) migrate to an SSH enabled server.  Let me know if any of you think this would be an option/solution...   :)
Forum Netiquette

"No matter how smart you are you can never convince someone stupid that they are stupid."  Anonymous

VastOne

  •      v-ger
  • Posts: 3794
    • View Profile
    • VSIDO Community
Re: FTP Script that only sends half the files
« Reply #10 on: February 24, 2016, 02:54:40 PM »
The other option may be to compress the iso files.

For what it's worth, I downloaded all 4 files then used both ftp manually and your script to upload them to the ftp server I run on the desktop computer. Both test uploads went to completion and the md5sums checked out.

Thanks for that testing and reporting back.  I am not sure how much compression we could get since it is maxed xz compressed already

Hey all, I can (and will, if needed) migrate to an SSH enabled server.  Let me know if any of you think this would be an option/solution...   :)

Hopefully we do not need that extreme and can find a working solution... gFTP does work fine and I can have multiple scripts that send each ISO separately.  Uploading the ISO files has always been a hands on affair

Thanks for the offer I do appreciate it  ...   8)
VSIDO      VSIDO Twitter     VSIDO FB     VSIDO Google+   

I dev VSIDO

hakerdefo

  • Posts: 480
    • View Profile
    • Looking at Linux through the Windows of Life
Re: FTP Script that only sends half the files
« Reply #11 on: March 06, 2016, 10:47:25 PM »
Give the following a try and let me know  8)

vsftup - VSIDO FTP Uploader

Cheers!!!
You Can't Always Git What You Want

hakerdefo

  • Posts: 480
    • View Profile
    • Looking at Linux through the Windows of Life
Re: FTP Script that only sends half the files
« Reply #12 on: March 09, 2016, 11:03:36 PM »
Yet Another FTP uploader for VSIDO,

vsncftp - yet another FTP uploader

Cheers!!!
You Can't Always Git What You Want