File path for attachments outside of wp-content/uploads

I am creating some custom post creation pages with PHP.

I create a simple post and want to attach an image I already have on my server to the post without moving it into the upload directory. I am trying to do it this way.

$post=array(
            'post_content'=>$body,//
            'post_title'=>$title,//
            'post_status'=>"publish",//
            'post_author'=>1,//
            'post_category'=>Array($category),//
            );

$newPostID = wp_insert_post($post,$wp_error);

$attachment = array(
                    'guid' => $imageURL,
                    'post_mime_type' => $filetype['type'],
                    'post_title' => preg_replace( '/\.[^.]+$/', '', basename( $imageURL ) ),
                    'post_content' => '',
                    'post_status' => 'inherit'
                    );
print_r($attachment);
$attach_id = wp_insert_attachment($attachment, $imageURL, $newPostID);
set_post_thumbnail($newPostID,$attach_id);

Where $imageURL is the path to my image file.

When I do this and check the Media Library, there is an entry for this file, but for its URL, it has my upload directory prefixed to $imageURL. Like http://domain/uploads/http://domain/myRealImageFolder

Is there away to achieve what I want without copying the file into the upload folder?